In [117]:
plt.figure(figsize=(15,5))
ax = sns.countplot(data["Age"],palette='winter')
perc_on_bar(ax,data["Age"])
AllLife Bank is a US bank that has a growing customer base. The majority of these customers are liability customers (depositors) with varying sizes of deposits. The number of customers who are also borrowers (asset customers) is quite small, and the bank is interested in expanding this base rapidly to bring in more loan business and in the process, earn more through the interest on loans. In particular, the management wants to explore ways of converting its liability customers to personal loan customers (while retaining them as depositors).
A campaign that the bank ran last year for liability customers showed a healthy conversion rate of over 9% success. This has encouraged the retail marketing department to devise campaigns with better target marketing to increase the success ratio.
You as a Data scientist at AllLife bank have to build a model that will help the marketing department to identify the potential customers who have a higher probability of purchasing the loan.
import warnings
warnings.filterwarnings("ignore")
# Libraries to help with reading and manipulating data
import pandas as pd
import numpy as np
# Library to split data
from sklearn.model_selection import train_test_split
# Libraries to help with model building
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree
from sklearn.linear_model import LinearRegression
# Libaries to help with data visualization
import matplotlib.pyplot as plt
import seaborn as sns
# Removes the limit from the number of displayed columns and rows.
pd.set_option("display.max_columns", None)
# pd.set_option('display.max_rows', None)
pd.set_option("display.max_rows", 200)
# for statistical analysis
import statsmodels.stats.api as sms
from statsmodels.stats.outliers_influence import variance_inflation_factor
from statsmodels.tools.tools import add_constant
# To get diferent metric scores
from sklearn import metrics
from sklearn.metrics import accuracy_score, recall_score, precision_score, roc_auc_score, roc_curve, confusion_matrix, precision_recall_curve, f1_score
from sklearn.model_selection import GridSearchCV
data=pd.read_csv( r'C:\Users\robby\Downloads\Loan_Modelling.csv')
# copying data to another varaible to avoid any changes to original data
data=data.copy()
data.head(10)
| ID | Age | Experience | Income | ZIPCode | Family | CCAvg | Education | Mortgage | Personal_Loan | Securities_Account | CD_Account | Online | CreditCard | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 25 | 1 | 49 | 91107 | 4 | 1.6 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| 1 | 2 | 45 | 19 | 34 | 90089 | 3 | 1.5 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| 2 | 3 | 39 | 15 | 11 | 94720 | 1 | 1.0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 4 | 35 | 9 | 100 | 94112 | 1 | 2.7 | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | 5 | 35 | 8 | 45 | 91330 | 4 | 1.0 | 2 | 0 | 0 | 0 | 0 | 0 | 1 |
| 5 | 6 | 37 | 13 | 29 | 92121 | 4 | 0.4 | 2 | 155 | 0 | 0 | 0 | 1 | 0 |
| 6 | 7 | 53 | 27 | 72 | 91711 | 2 | 1.5 | 2 | 0 | 0 | 0 | 0 | 1 | 0 |
| 7 | 8 | 50 | 24 | 22 | 93943 | 1 | 0.3 | 3 | 0 | 0 | 0 | 0 | 0 | 1 |
| 8 | 9 | 35 | 10 | 81 | 90089 | 3 | 0.6 | 2 | 104 | 0 | 0 | 0 | 1 | 0 |
| 9 | 10 | 34 | 9 | 180 | 93023 | 1 | 8.9 | 3 | 0 | 1 | 0 | 0 | 0 | 0 |
data.tail(10)
| ID | Age | Experience | Income | ZIPCode | Family | CCAvg | Education | Mortgage | Personal_Loan | Securities_Account | CD_Account | Online | CreditCard | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 4990 | 4991 | 55 | 25 | 58 | 95023 | 4 | 2.00 | 3 | 219 | 0 | 0 | 0 | 0 | 1 |
| 4991 | 4992 | 51 | 25 | 92 | 91330 | 1 | 1.90 | 2 | 100 | 0 | 0 | 0 | 0 | 1 |
| 4992 | 4993 | 30 | 5 | 13 | 90037 | 4 | 0.50 | 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4993 | 4994 | 45 | 21 | 218 | 91801 | 2 | 6.67 | 1 | 0 | 0 | 0 | 0 | 1 | 0 |
| 4994 | 4995 | 64 | 40 | 75 | 94588 | 3 | 2.00 | 3 | 0 | 0 | 0 | 0 | 1 | 0 |
| 4995 | 4996 | 29 | 3 | 40 | 92697 | 1 | 1.90 | 3 | 0 | 0 | 0 | 0 | 1 | 0 |
| 4996 | 4997 | 30 | 4 | 15 | 92037 | 4 | 0.40 | 1 | 85 | 0 | 0 | 0 | 1 | 0 |
| 4997 | 4998 | 63 | 39 | 24 | 93023 | 2 | 0.30 | 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4998 | 4999 | 65 | 40 | 49 | 90034 | 3 | 0.50 | 2 | 0 | 0 | 0 | 0 | 1 | 0 |
| 4999 | 5000 | 28 | 4 | 83 | 92612 | 3 | 0.80 | 1 | 0 | 0 | 0 | 0 | 1 | 1 |
data.shape
(5000, 14)
data.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 5000 entries, 0 to 4999 Data columns (total 14 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 ID 5000 non-null int64 1 Age 5000 non-null int64 2 Experience 5000 non-null int64 3 Income 5000 non-null int64 4 ZIPCode 5000 non-null int64 5 Family 5000 non-null int64 6 CCAvg 5000 non-null float64 7 Education 5000 non-null int64 8 Mortgage 5000 non-null int64 9 Personal_Loan 5000 non-null int64 10 Securities_Account 5000 non-null int64 11 CD_Account 5000 non-null int64 12 Online 5000 non-null int64 13 CreditCard 5000 non-null int64 dtypes: float64(1), int64(13) memory usage: 547.0 KB
data.isnull().sum()
ID 0 Age 0 Experience 0 Income 0 ZIPCode 0 Family 0 CCAvg 0 Education 0 Mortgage 0 Personal_Loan 0 Securities_Account 0 CD_Account 0 Online 0 CreditCard 0 dtype: int64
data[data.duplicated()].count()
ID 0 Age 0 Experience 0 Income 0 ZIPCode 0 Family 0 CCAvg 0 Education 0 Mortgage 0 Personal_Loan 0 Securities_Account 0 CD_Account 0 Online 0 CreditCard 0 dtype: int64
data.nunique()
ID 5000 Age 45 Experience 47 Income 162 ZIPCode 467 Family 4 CCAvg 108 Education 3 Mortgage 347 Personal_Loan 2 Securities_Account 2 CD_Account 2 Online 2 CreditCard 2 dtype: int64
data.describe().T
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| ID | 5000.0 | 2500.500000 | 1443.520003 | 1.0 | 1250.75 | 2500.5 | 3750.25 | 5000.0 |
| Age | 5000.0 | 45.338400 | 11.463166 | 23.0 | 35.00 | 45.0 | 55.00 | 67.0 |
| Experience | 5000.0 | 20.104600 | 11.467954 | -3.0 | 10.00 | 20.0 | 30.00 | 43.0 |
| Income | 5000.0 | 73.774200 | 46.033729 | 8.0 | 39.00 | 64.0 | 98.00 | 224.0 |
| ZIPCode | 5000.0 | 93169.257000 | 1759.455086 | 90005.0 | 91911.00 | 93437.0 | 94608.00 | 96651.0 |
| Family | 5000.0 | 2.396400 | 1.147663 | 1.0 | 1.00 | 2.0 | 3.00 | 4.0 |
| CCAvg | 5000.0 | 1.937938 | 1.747659 | 0.0 | 0.70 | 1.5 | 2.50 | 10.0 |
| Education | 5000.0 | 1.881000 | 0.839869 | 1.0 | 1.00 | 2.0 | 3.00 | 3.0 |
| Mortgage | 5000.0 | 56.498800 | 101.713802 | 0.0 | 0.00 | 0.0 | 101.00 | 635.0 |
| Personal_Loan | 5000.0 | 0.096000 | 0.294621 | 0.0 | 0.00 | 0.0 | 0.00 | 1.0 |
| Securities_Account | 5000.0 | 0.104400 | 0.305809 | 0.0 | 0.00 | 0.0 | 0.00 | 1.0 |
| CD_Account | 5000.0 | 0.060400 | 0.238250 | 0.0 | 0.00 | 0.0 | 0.00 | 1.0 |
| Online | 5000.0 | 0.596800 | 0.490589 | 0.0 | 0.00 | 1.0 | 1.00 | 1.0 |
| CreditCard | 5000.0 | 0.294000 | 0.455637 | 0.0 | 0.00 | 0.0 | 1.00 | 1.0 |
def perc_on_bar(plot, feature):
'''
plot
feature: categorical feature
the function won't work if a column is passed in hue parameter
'''
total = len(feature) # length of the column
for p in ax.patches:
percentage = '{:.1f}%'.format(100 * p.get_height()/total) # percentage of each class of the category
x = p.get_x() + p.get_width() / 2 - 0.05 # width of the plot
y = p.get_y() + p.get_height() # hieght of the plot
ax.annotate(percentage, (x, y), size = 12) # annotate the percantage
plt.show() # show the plot
plt.figure(figsize=(15,5))
ax = sns.countplot(data["Age"],palette='winter')
perc_on_bar(ax,data["Age"])
plt.figure(figsize=(15,5))
ax = sns.countplot(data["Experience"],palette='winter')
perc_on_bar(ax,data["Experience"])
plt.figure(figsize=(65,45))
ax = sns.countplot(data["Income"],palette='winter')
perc_on_bar(ax,data["Income"])
plt.figure(figsize=(65,35))
ax = sns.countplot(data["ZIPCode"],palette='winter')
perc_on_bar(ax,data["ZIPCode"])
plt.figure(figsize=(15,5))
ax = sns.countplot(data["Family"],palette='winter')
perc_on_bar(ax,data["Family"])
plt.figure(figsize=(45,15))
ax = sns.countplot(data["CCAvg"],palette='winter')
perc_on_bar(ax,data["CCAvg"])
plt.figure(figsize=(15,5))
ax = sns.countplot(data["Education"],palette='winter')
perc_on_bar(ax,data["Education"])
plt.figure(figsize=(25,5))
ax = sns.countplot(data["Mortgage"],palette='winter')
perc_on_bar(ax,data["Mortgage"])
plt.figure(figsize=(15,5))
ax = sns.countplot(data["Personal_Loan"],palette='winter')
perc_on_bar(ax,data["Personal_Loan"])
plt.figure(figsize=(15,7))
sns.heatmap(data.corr(),annot=True)
plt.show()
sns.pairplot(data=data,hue="Personal_Loan",)
plt.show()
## Function to plot stacked bar chart
def stacked_plot(x):
sns.set(palette='nipy_spectral')
tab1 = pd.crosstab(x,data['Personal_Loan'],margins=True)
print(tab1)
print('-'*120)
tab = pd.crosstab(x,data['Personal_Loan'],normalize='index')
tab.plot(kind='bar',stacked=True,figsize=(10,5))
#plt.legend(loc='lower left', frameon=False)
#plt.legend(loc="upper left", bbox_to_anchor=(0,1))
plt.show()
stacked_plot(data['Age'])
Personal_Loan 0 1 All Age 23 12 0 12 24 28 0 28 25 53 0 53 26 65 13 78 27 79 12 91 28 94 9 103 29 108 15 123 30 119 17 136 31 118 7 125 32 108 12 120 33 105 15 120 34 116 18 134 35 135 16 151 36 91 16 107 37 98 8 106 38 103 12 115 39 127 6 133 40 117 8 125 41 128 8 136 42 112 14 126 43 134 15 149 44 107 14 121 45 114 13 127 46 114 13 127 47 103 10 113 48 106 12 118 49 105 10 115 50 125 13 138 51 119 10 129 52 130 15 145 53 101 11 112 54 128 15 143 55 116 9 125 56 121 14 135 57 120 12 132 58 133 10 143 59 123 9 132 60 117 10 127 61 110 12 122 62 114 9 123 63 92 16 108 64 70 8 78 65 66 14 80 66 24 0 24 67 12 0 12 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['Mortgage'])
Personal_Loan 0 1 All Mortgage 0 3150 312 3462 75 7 1 8 76 11 1 12 77 4 0 4 78 15 0 15 ... ... ... ... 601 1 0 1 612 0 1 1 617 0 1 1 635 1 0 1 All 4520 480 5000 [348 rows x 3 columns] ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['Experience'])
Personal_Loan 0 1 All Experience -3 4 0 4 -2 15 0 15 -1 33 0 33 0 59 7 66 1 66 8 74 2 76 9 85 3 112 17 129 4 104 9 113 5 132 14 146 6 107 12 119 7 109 12 121 8 101 18 119 9 127 20 147 10 111 7 118 11 103 13 116 12 86 16 102 13 106 11 117 14 121 6 127 15 114 5 119 16 114 13 127 17 114 11 125 18 125 12 137 19 121 14 135 20 131 17 148 21 102 11 113 22 111 13 124 23 131 13 144 24 123 8 131 25 128 14 142 26 120 14 134 27 115 10 125 28 127 11 138 29 112 12 124 30 113 13 126 31 92 12 104 32 140 14 154 33 110 7 117 34 115 10 125 35 130 13 143 36 102 12 114 37 103 13 116 38 80 8 88 39 75 10 85 40 53 4 57 41 36 7 43 42 8 0 8 43 3 0 3 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['Income'])
Personal_Loan 0 1 All Income 8 23 0 23 9 26 0 26 10 23 0 23 11 27 0 27 12 30 0 30 13 32 0 32 14 31 0 31 15 33 0 33 18 53 0 53 19 52 0 52 20 47 0 47 21 65 0 65 22 65 0 65 23 54 0 54 24 47 0 47 25 64 0 64 28 63 0 63 29 67 0 67 30 63 0 63 31 55 0 55 32 58 0 58 33 51 0 51 34 53 0 53 35 65 0 65 38 84 0 84 39 81 0 81 40 78 0 78 41 82 0 82 42 77 0 77 43 70 0 70 44 85 0 85 45 69 0 69 48 44 0 44 49 52 0 52 50 45 0 45 51 41 0 41 52 47 0 47 53 57 0 57 54 52 0 52 55 61 0 61 58 55 0 55 59 53 0 53 60 51 1 52 61 57 0 57 62 55 0 55 63 46 0 46 64 59 1 60 65 59 1 60 68 35 0 35 69 45 1 46 70 47 0 47 71 42 1 43 72 41 0 41 73 43 1 44 74 45 0 45 75 46 1 47 78 61 0 61 79 53 0 53 80 56 0 56 81 82 1 83 82 60 1 61 83 67 7 74 84 62 1 63 85 63 2 65 88 26 0 26 89 32 2 34 90 37 1 38 91 35 2 37 92 28 1 29 93 33 4 37 94 24 2 26 95 22 3 25 98 26 2 28 99 19 5 24 100 9 1 10 101 20 4 24 102 13 3 16 103 14 4 18 104 17 3 20 105 17 3 20 108 12 4 16 109 15 3 18 110 16 3 19 111 15 7 22 112 23 3 26 113 29 5 34 114 23 7 30 115 19 8 27 118 13 6 19 119 13 5 18 120 11 6 17 121 18 2 20 122 17 7 24 123 9 9 18 124 9 3 12 125 16 7 23 128 20 4 24 129 15 8 23 130 8 11 19 131 11 8 19 132 11 7 18 133 8 7 15 134 13 7 20 135 8 10 18 138 13 5 18 139 10 6 16 140 13 6 19 141 15 9 24 142 7 8 15 143 5 4 9 144 5 2 7 145 17 6 23 148 7 4 11 149 16 4 20 150 9 2 11 151 3 1 4 152 10 5 15 153 7 4 11 154 12 9 21 155 14 5 19 158 8 10 18 159 3 4 7 160 8 4 12 161 9 7 16 162 7 3 10 163 7 2 9 164 6 7 13 165 5 6 11 168 2 6 8 169 1 6 7 170 4 8 12 171 5 4 9 172 3 8 11 173 5 8 13 174 4 5 9 175 7 5 12 178 4 6 10 179 8 9 17 180 10 8 18 181 4 4 8 182 2 11 13 183 6 6 12 184 3 9 12 185 3 6 9 188 3 7 10 189 1 1 2 190 4 7 11 191 6 7 13 192 2 4 6 193 2 4 6 194 4 4 8 195 10 5 15 198 3 0 3 199 3 0 3 200 3 0 3 201 4 1 5 202 1 1 2 203 1 1 2 204 3 0 3 205 2 0 2 218 1 0 1 224 1 0 1 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['Family'])
Personal_Loan 0 1 All Family 1 1365 107 1472 2 1190 106 1296 3 877 133 1010 4 1088 134 1222 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['Education'])
Personal_Loan 0 1 All Education 1 2003 93 2096 2 1221 182 1403 3 1296 205 1501 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['CCAvg'])
Personal_Loan 0 1 All CCAvg 0.0 105 1 106 0.1 181 2 183 0.2 196 8 204 0.3 235 6 241 0.4 175 4 179 0.5 155 8 163 0.6 114 4 118 0.67 18 0 18 0.7 163 6 169 0.75 9 0 9 0.8 182 5 187 0.9 103 3 106 1.0 229 2 231 1.1 77 7 84 1.2 60 6 66 1.3 121 7 128 1.33 9 0 9 1.4 131 5 136 1.5 174 4 178 1.6 98 3 101 1.67 18 0 18 1.7 154 4 158 1.75 9 0 9 1.8 149 3 152 1.9 102 4 106 2.0 184 4 188 2.1 97 3 100 2.2 123 7 130 2.3 53 5 58 2.33 18 0 18 2.4 87 5 92 2.5 105 2 107 2.6 79 8 87 2.67 36 0 36 2.7 51 7 58 2.75 0 1 1 2.8 105 5 110 2.9 45 9 54 3.0 34 19 53 3.1 8 12 20 3.2 17 5 22 3.25 0 1 1 3.3 35 10 45 3.33 0 1 1 3.4 26 13 39 3.5 9 6 15 3.6 17 10 27 3.67 0 1 1 3.7 18 7 25 3.8 33 10 43 3.9 18 9 27 4.0 26 7 33 4.1 9 13 22 4.2 0 11 11 4.25 0 2 2 4.3 18 8 26 4.33 9 0 9 4.4 9 8 17 4.5 25 4 29 4.6 8 6 14 4.67 0 1 1 4.7 17 7 24 4.75 0 2 2 4.8 0 7 7 4.9 17 5 22 5.0 9 9 18 5.1 0 6 6 5.2 9 7 16 5.3 0 4 4 5.33 0 1 1 5.4 8 10 18 5.5 0 4 4 5.6 0 7 7 5.67 0 2 2 5.7 8 5 13 5.8 0 3 3 5.9 0 5 5 6.0 18 8 26 6.1 8 6 14 6.2 0 2 2 6.3 8 5 13 6.33 9 1 10 6.4 0 3 3 6.5 8 10 18 6.6 0 4 4 6.67 9 0 9 6.7 9 0 9 6.8 8 2 10 6.9 9 5 14 7.0 9 5 14 7.2 9 4 13 7.3 9 1 10 7.4 9 4 13 7.5 9 3 12 7.6 9 0 9 7.8 9 0 9 7.9 0 4 4 8.0 9 3 12 8.1 9 1 10 8.2 0 1 1 8.3 0 2 2 8.5 0 2 2 8.6 8 0 8 8.8 8 1 9 8.9 0 1 1 9.0 0 2 2 9.3 0 1 1 10.0 0 3 3 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['Securities_Account'])
Personal_Loan 0 1 All Securities_Account 0 4058 420 4478 1 462 60 522 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['CD_Account'])
Personal_Loan 0 1 All CD_Account 0 4358 340 4698 1 162 140 302 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['Online'])
Personal_Loan 0 1 All Online 0 1827 189 2016 1 2693 291 2984 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
stacked_plot(data['CreditCard'])
Personal_Loan 0 1 All CreditCard 0 3193 337 3530 1 1327 143 1470 All 4520 480 5000 ------------------------------------------------------------------------------------------------------------------------
## Function to plot stacked bar chart
def stacked_plot(x):
sns.set(palette='nipy_spectral')
tab1 = pd.crosstab(x,data['CCAvg'],margins=True)
print(tab1)
print('-'*200)
tab = pd.crosstab(x,data['CCAvg'],normalize='index')
tab.plot(kind='bar',stacked=True,figsize= (20,10))
#plt.legend(loc='lower left', frameon=False)
#plt.legend(loc="upper left", bbox_to_anchor=(0,1))
plt.show()
stacked_plot(data['Income'])
CCAvg 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.67 0.7 0.75 0.8 0.9 1.0 \ Income 8 0 5 2 4 3 1 1 1 3 0 1 1 1 9 3 3 5 5 0 3 1 0 3 0 0 0 3 10 1 3 1 2 5 2 0 1 5 0 1 0 2 11 2 5 5 3 2 2 1 1 1 0 1 1 3 12 1 3 5 3 3 2 2 1 2 0 0 0 8 13 2 1 2 4 4 3 1 1 3 0 1 6 4 14 1 5 1 2 5 2 1 0 3 0 2 3 6 15 1 4 3 5 6 0 4 0 2 0 4 2 2 18 1 5 4 12 10 4 2 0 0 0 1 3 2 19 1 3 4 4 9 6 1 2 4 2 1 3 2 20 1 3 4 6 7 6 1 1 2 1 2 0 6 21 1 9 8 4 10 4 4 3 5 0 2 2 8 22 1 9 6 7 7 7 4 1 2 0 1 2 6 23 1 4 9 9 8 3 1 3 2 0 1 2 4 24 0 4 7 6 8 3 1 1 2 0 2 3 5 25 1 8 4 7 6 3 3 2 3 0 3 6 15 28 0 1 5 8 1 4 3 0 4 1 7 1 7 29 0 4 4 9 2 4 4 0 3 1 3 2 7 30 1 3 4 5 5 4 3 0 7 0 3 1 10 31 0 2 2 10 10 3 1 0 0 0 1 2 6 32 1 3 2 5 4 2 6 0 3 1 2 0 7 33 1 3 2 4 4 4 0 0 3 2 1 2 4 34 1 4 4 6 4 0 4 0 5 0 0 2 1 35 1 3 6 6 2 4 2 0 2 1 5 2 5 38 0 2 4 4 0 3 4 0 3 0 3 6 10 39 1 1 5 3 1 3 2 0 4 0 7 3 5 40 1 2 1 5 2 3 3 0 0 0 3 4 8 41 0 2 4 3 1 5 3 0 0 0 3 2 8 42 0 3 2 4 1 4 2 0 7 0 2 0 3 43 0 4 4 5 1 1 0 0 4 0 1 2 4 44 1 2 8 9 0 2 5 0 2 0 5 4 9 45 1 5 4 1 0 1 7 0 5 0 2 2 4 48 0 1 2 3 0 0 1 0 1 0 2 0 2 49 0 0 0 0 0 3 0 0 1 0 2 1 0 50 1 1 0 0 0 1 2 0 0 0 1 0 1 51 0 0 2 1 0 0 1 0 1 0 0 0 3 52 0 3 1 0 0 4 0 0 2 0 2 0 3 53 1 1 1 1 0 2 0 0 2 0 4 0 2 54 0 1 1 3 0 1 3 0 3 0 2 1 1 55 0 1 3 1 0 1 0 0 3 0 1 5 0 58 3 1 2 1 1 1 1 0 2 0 2 2 1 59 3 1 3 0 1 4 0 0 0 0 4 2 0 60 1 0 2 0 1 2 1 0 2 0 0 0 0 61 0 0 2 0 1 3 1 0 2 0 1 4 0 62 2 2 1 2 0 0 0 0 4 0 4 2 2 63 0 0 1 0 0 3 0 0 2 0 4 0 2 64 2 3 1 2 1 2 0 0 1 0 3 2 1 65 4 1 2 0 1 4 0 0 4 0 0 0 1 68 3 0 2 0 0 0 0 0 1 0 2 0 0 69 0 4 0 3 1 1 0 0 4 0 4 1 1 70 1 1 1 3 0 1 0 0 1 0 1 0 0 71 0 4 1 4 0 0 0 0 1 0 2 0 0 72 2 1 1 4 0 0 0 0 4 0 2 0 0 73 0 0 0 1 1 0 0 0 2 0 4 0 0 74 1 2 0 1 1 0 0 0 6 0 0 1 0 75 2 3 2 2 0 0 0 0 2 0 0 1 0 78 1 2 4 0 1 1 2 0 1 0 2 0 2 79 1 3 2 1 0 0 0 0 3 0 4 0 1 80 1 2 1 1 2 3 0 0 2 0 4 0 0 81 4 2 4 1 4 0 1 0 1 0 4 2 0 82 2 2 1 0 3 0 0 0 1 0 3 2 1 83 1 3 2 1 3 1 0 0 2 0 3 0 2 84 2 4 2 0 1 0 2 0 4 0 2 0 0 85 1 3 3 0 3 0 0 0 0 0 1 0 0 88 1 0 0 1 0 0 0 0 0 0 3 0 2 89 1 3 0 0 1 3 0 0 0 0 5 0 1 90 3 1 0 3 0 1 0 0 0 0 4 0 0 91 3 1 1 0 0 1 0 0 1 0 3 0 1 92 4 0 2 0 0 1 1 0 1 0 0 0 0 93 1 0 2 1 0 0 0 0 1 0 4 0 0 94 1 0 1 1 0 3 0 0 0 0 2 0 1 95 2 1 1 2 0 2 0 0 1 0 1 0 1 98 0 1 0 0 3 0 0 0 0 0 0 1 1 99 0 1 0 0 0 0 0 0 0 0 0 0 1 100 0 0 0 0 0 0 1 0 0 0 1 0 0 101 0 0 0 1 1 0 2 0 0 0 0 0 0 102 0 0 0 1 0 0 0 0 0 0 0 0 0 103 0 0 0 1 0 0 1 0 0 0 2 0 1 104 0 0 0 0 0 0 1 0 0 0 0 0 3 105 0 0 0 0 0 0 0 0 0 0 2 0 0 108 0 0 0 0 0 0 0 0 0 0 0 1 1 109 0 0 0 0 0 1 1 0 0 0 0 0 1 110 0 0 1 0 0 0 0 0 0 0 0 0 0 111 1 1 1 1 1 0 0 0 0 0 1 0 0 112 1 0 0 1 0 0 1 0 0 0 0 0 0 113 0 2 4 0 1 0 1 0 0 0 2 1 3 114 1 0 1 2 1 0 3 0 0 0 3 0 1 115 0 0 0 1 1 1 0 0 0 0 1 0 1 118 0 0 0 1 0 0 0 0 0 0 0 0 0 119 0 0 0 0 0 0 1 0 0 0 0 0 0 120 2 0 0 0 0 0 1 0 0 0 1 0 0 121 0 0 0 1 1 0 0 0 1 0 0 0 0 122 2 0 3 4 0 1 0 0 0 0 0 0 0 123 0 0 0 0 0 0 1 0 0 0 1 0 0 124 0 0 2 1 0 0 1 0 0 0 0 0 0 125 2 0 0 0 0 1 2 0 0 0 0 0 1 128 0 0 0 0 2 0 1 0 0 0 0 1 0 129 0 0 0 2 0 0 0 0 1 0 0 1 0 130 0 0 0 0 0 0 1 0 0 0 0 0 1 131 0 0 0 1 0 1 1 0 1 0 0 1 0 132 0 0 0 3 0 0 1 0 0 0 0 0 0 133 1 0 0 0 0 0 1 0 0 0 0 0 0 134 1 0 0 1 0 0 0 0 0 0 0 1 0 135 0 0 0 1 0 0 2 0 0 0 0 0 0 138 2 0 0 0 0 0 0 0 0 0 0 0 0 139 1 0 0 0 1 0 0 0 0 0 1 1 0 140 0 0 0 0 0 3 0 0 0 0 0 1 0 141 1 0 1 0 0 1 0 0 1 0 1 0 0 142 1 0 0 0 0 0 0 0 0 0 2 0 0 143 0 0 0 0 0 0 0 0 0 0 0 0 0 144 0 0 0 0 0 0 0 0 0 0 0 0 0 145 0 0 0 2 0 1 0 0 0 0 1 1 0 148 0 0 0 0 1 0 0 0 0 0 0 0 1 149 0 0 1 0 2 0 0 0 0 0 1 0 0 150 0 0 0 0 0 0 0 0 0 0 1 0 0 151 0 0 0 0 0 0 1 0 0 0 0 0 0 152 1 0 0 0 0 0 0 0 0 0 0 0 0 153 0 0 0 0 0 0 0 0 0 0 0 0 0 154 0 0 0 0 0 0 0 0 0 0 0 0 0 155 0 1 0 0 1 0 0 0 0 0 1 0 1 158 0 0 0 0 2 0 0 0 0 0 0 0 0 159 0 0 0 0 0 2 0 0 0 0 0 0 0 160 0 0 0 0 0 0 0 0 0 0 0 0 0 161 0 0 0 0 0 0 0 0 0 0 0 0 0 162 0 0 0 0 0 0 0 0 0 0 0 0 0 163 0 0 0 0 2 1 0 0 0 0 0 0 0 164 1 0 0 0 0 1 0 0 0 0 0 0 0 165 0 0 0 0 0 0 0 0 0 0 0 0 0 168 0 0 0 0 0 0 0 0 0 0 0 0 0 169 0 0 0 0 0 0 0 0 0 0 0 0 0 170 0 1 0 0 0 0 0 0 0 0 0 0 1 171 0 0 0 0 0 0 0 0 1 0 0 0 0 172 0 0 0 0 0 0 0 0 0 0 0 0 1 173 0 0 1 0 0 1 0 0 0 0 0 0 1 174 0 0 0 0 0 0 0 0 0 0 0 0 0 175 0 0 0 0 0 1 0 0 1 0 0 0 0 178 0 0 0 0 0 0 0 0 0 0 1 0 1 179 3 0 0 0 0 0 0 0 0 0 0 0 1 180 0 0 0 0 0 0 0 0 0 0 0 0 1 181 0 0 0 0 0 0 0 0 0 0 0 0 0 182 0 0 0 1 0 0 0 0 0 0 1 0 0 183 1 0 0 0 0 0 0 0 0 0 0 0 1 184 0 0 0 0 0 0 0 0 0 0 0 0 0 185 0 0 0 0 0 0 0 0 0 0 0 0 0 188 0 0 0 0 0 0 0 0 0 0 0 1 0 189 0 0 0 0 0 0 0 0 0 0 0 0 0 190 0 0 0 1 0 0 0 0 0 0 0 0 0 191 0 0 0 0 0 0 0 0 0 0 0 0 0 192 1 0 0 0 0 0 0 0 0 0 0 0 0 193 0 0 0 0 0 0 0 0 0 0 0 0 0 194 1 0 1 0 0 0 0 0 0 0 0 0 0 195 0 0 0 0 1 0 0 0 0 0 0 0 0 198 0 0 0 0 0 0 0 0 0 0 0 0 0 199 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 201 0 0 0 0 0 0 0 0 0 0 0 0 0 202 0 0 0 0 0 0 0 0 0 0 0 0 0 203 0 0 0 0 0 0 0 0 0 0 0 0 0 204 0 0 0 0 0 0 0 0 0 0 0 0 0 205 0 0 0 0 0 0 0 0 0 0 0 0 0 218 0 0 0 0 0 0 0 0 0 0 0 0 0 224 0 0 0 0 0 0 0 0 0 0 0 0 0 All 106 183 204 241 179 163 118 18 169 9 187 106 231 CCAvg 1.1 1.2 1.3 1.33 1.4 1.5 1.6 1.67 1.7 1.75 1.8 1.9 2.0 \ Income 8 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 1 3 0 0 5 0 0 0 0 0 0 0 19 1 1 2 0 3 3 0 0 0 0 0 0 0 20 1 0 2 0 3 1 0 0 0 0 0 0 0 21 0 0 2 0 2 1 0 0 0 0 0 0 0 22 0 2 2 0 3 5 0 0 0 0 0 0 0 23 1 1 1 0 2 2 0 0 0 0 0 0 0 24 1 0 1 0 0 3 0 0 0 0 0 0 0 25 0 0 0 0 2 1 0 0 0 0 0 0 0 28 1 2 2 0 3 8 0 0 3 0 0 0 2 29 1 0 4 1 3 8 0 0 0 0 1 2 4 30 0 0 5 0 4 2 0 0 3 0 0 1 2 31 3 1 3 0 2 5 0 0 1 0 2 0 1 32 3 0 1 1 5 3 0 0 1 0 2 2 4 33 0 1 0 0 4 7 0 0 2 0 1 1 5 34 1 0 4 0 0 6 0 0 3 0 1 1 6 35 0 2 6 1 4 5 0 0 4 0 2 2 0 38 3 0 4 2 4 4 2 0 5 0 1 3 4 39 1 2 3 0 2 5 1 0 4 0 2 6 5 40 5 2 3 0 2 0 2 2 2 0 1 3 5 41 3 2 4 1 4 4 0 1 5 0 3 3 7 42 2 2 3 1 1 7 2 0 3 0 3 2 3 43 4 5 5 0 3 3 3 1 4 0 2 5 1 44 2 2 1 1 1 4 3 1 2 0 3 5 5 45 1 1 6 1 2 3 2 0 4 0 6 0 4 48 0 1 2 0 0 2 3 2 0 1 2 1 0 49 2 2 0 0 3 3 2 3 3 0 3 1 1 50 2 0 1 0 2 4 0 1 3 1 3 1 2 51 2 0 2 0 3 2 3 0 2 1 2 2 3 52 4 1 1 0 2 1 1 1 3 0 1 1 3 53 1 1 0 0 1 1 5 2 2 0 3 4 2 54 2 2 2 0 0 2 4 1 2 0 4 1 1 55 5 0 2 0 3 2 4 3 5 0 2 1 7 58 0 1 4 0 0 1 0 0 2 0 4 0 7 59 0 2 0 0 2 3 4 0 2 1 0 2 1 60 0 2 2 0 1 0 3 0 2 0 2 1 2 61 0 0 1 0 2 1 2 0 4 0 2 1 5 62 0 1 1 0 2 3 1 0 0 1 5 0 1 63 0 2 0 0 1 4 6 0 1 1 1 0 4 64 0 1 0 0 0 4 1 0 4 0 5 1 1 65 0 2 1 0 0 5 0 0 2 0 2 0 2 68 0 0 1 0 1 4 2 0 0 0 2 0 1 69 0 0 1 0 1 2 1 0 2 0 2 1 1 70 1 1 1 0 4 2 3 0 0 1 0 3 3 71 0 0 1 0 4 0 0 0 3 2 4 3 2 72 1 0 0 0 3 1 2 0 2 0 1 0 3 73 1 0 0 0 1 2 1 0 2 0 1 0 2 74 1 1 1 0 2 0 1 0 0 0 2 2 2 75 1 0 0 0 3 2 0 0 2 0 2 3 2 78 1 1 0 0 1 0 1 0 2 0 4 1 7 79 2 0 0 0 1 1 2 0 3 0 2 0 5 80 1 1 2 0 0 2 6 0 3 0 4 0 2 81 0 1 1 0 1 2 2 0 4 0 6 1 3 82 0 0 3 0 1 3 1 0 2 0 4 0 2 83 0 0 0 0 2 2 1 0 3 0 6 0 7 84 3 1 2 0 0 1 4 0 0 0 6 1 4 85 2 3 3 0 0 0 5 0 3 0 4 4 3 88 1 0 0 0 2 0 4 0 1 0 2 1 0 89 0 0 0 0 3 1 0 0 2 0 0 5 1 90 2 0 1 0 1 0 1 0 0 0 1 4 0 91 0 0 0 0 2 1 2 0 2 0 1 1 1 92 2 0 1 0 1 0 1 0 0 0 0 3 2 93 0 0 0 0 0 1 3 0 1 0 1 1 0 94 3 0 0 0 0 1 0 0 0 0 1 2 1 95 0 0 0 0 0 0 1 0 2 0 1 2 0 98 1 1 2 0 0 0 0 0 0 0 5 0 2 99 1 0 0 0 2 1 0 0 2 0 2 1 0 100 0 1 0 0 0 0 0 0 2 0 0 0 1 101 0 1 0 0 0 0 0 0 1 0 0 1 1 102 0 0 0 0 1 0 0 0 1 0 0 2 1 103 0 0 0 0 0 0 0 0 0 0 1 1 1 104 0 1 0 0 0 0 0 0 0 0 3 0 0 105 0 1 0 0 1 0 0 0 4 0 0 1 0 108 0 1 0 0 0 0 0 0 0 0 1 1 1 109 1 0 0 0 0 0 0 0 2 0 2 0 2 110 0 0 0 0 0 1 0 0 1 0 3 0 1 111 1 1 1 0 1 1 0 0 0 0 0 0 0 112 0 0 0 0 1 0 1 0 1 0 3 0 3 113 0 0 1 0 1 0 0 0 3 0 1 0 4 114 0 0 0 0 0 0 0 0 1 0 1 0 1 115 0 1 1 0 0 1 0 0 1 0 0 2 1 118 0 0 1 0 0 0 0 0 0 0 0 1 1 119 0 0 0 0 0 1 0 0 0 0 0 0 2 120 0 0 0 0 0 0 0 0 0 0 0 0 1 121 1 0 1 0 0 1 0 0 0 0 0 0 2 122 0 0 2 0 0 0 0 0 0 0 0 0 0 123 0 0 1 0 0 0 1 0 0 0 0 0 0 124 0 0 0 0 0 0 0 0 0 0 0 0 0 125 0 0 1 0 0 0 0 0 0 0 0 1 0 128 0 0 0 0 0 1 0 0 0 0 0 0 0 129 0 0 2 0 0 1 0 0 0 0 0 1 0 130 2 0 1 0 1 0 0 0 0 0 0 0 0 131 0 1 0 0 0 0 0 0 0 0 0 0 0 132 1 1 0 0 0 0 0 0 0 0 0 0 0 133 0 0 0 0 1 1 0 0 0 0 0 0 1 134 0 0 0 0 0 0 0 0 1 0 0 0 0 135 0 0 0 0 1 2 0 0 0 0 0 0 0 138 0 0 0 0 0 0 0 0 0 0 0 0 1 139 0 0 0 0 0 0 0 0 0 0 0 0 1 140 0 0 0 0 0 0 0 0 0 0 0 1 1 141 0 0 0 0 0 0 0 0 0 0 0 0 0 142 0 0 0 0 1 1 0 0 0 0 0 0 0 143 0 0 0 0 0 0 0 0 1 0 0 0 0 144 0 0 0 0 0 0 0 0 0 0 0 0 0 145 0 0 0 0 0 0 0 0 1 0 1 0 0 148 0 0 0 0 0 1 0 0 0 0 0 0 0 149 0 0 0 0 0 1 0 0 1 0 0 0 0 150 0 0 0 0 0 0 0 0 0 0 0 0 1 151 0 0 0 0 0 0 0 0 0 0 0 0 0 152 0 0 0 0 2 0 0 0 0 0 0 0 0 153 0 0 1 0 0 0 0 0 1 0 0 0 2 154 0 0 0 0 0 0 0 0 1 0 0 0 1 155 0 0 1 0 0 1 0 0 0 0 0 0 0 158 0 0 0 0 0 0 0 0 0 0 0 0 0 159 0 0 0 0 0 1 0 0 0 0 0 1 0 160 0 0 0 0 0 0 0 0 0 0 0 0 0 161 0 0 0 0 0 0 0 0 1 0 0 0 0 162 0 0 3 0 1 0 0 0 0 0 0 0 0 163 0 0 1 0 0 0 0 0 0 0 0 0 0 164 0 0 1 0 0 0 0 0 0 0 0 0 0 165 0 0 0 0 0 0 1 0 0 0 0 0 0 168 0 0 1 0 0 0 0 0 0 0 0 0 0 169 0 0 0 0 0 0 0 0 0 0 0 0 0 170 0 0 0 0 0 0 0 0 0 0 0 0 0 171 0 0 0 0 1 0 0 0 0 0 0 1 0 172 0 0 0 0 0 0 0 0 0 0 0 0 0 173 0 0 0 0 1 1 0 0 0 0 0 0 0 174 0 0 0 0 0 0 0 0 1 0 0 0 0 175 2 0 0 0 1 0 0 0 0 0 0 0 0 178 0 0 0 0 0 0 0 0 0 0 0 0 0 179 0 1 0 0 0 0 0 0 0 0 0 0 0 180 0 0 0 0 0 0 0 0 3 0 0 0 0 181 0 0 0 0 1 0 0 0 1 0 0 0 0 182 0 1 0 0 1 0 0 0 0 0 0 0 0 183 0 0 0 0 2 0 0 0 0 0 0 0 0 184 0 0 0 0 0 0 0 0 0 0 0 0 0 185 0 0 0 0 0 0 0 0 1 0 0 0 1 188 0 0 1 0 0 0 0 0 0 0 0 0 0 189 0 0 0 0 0 0 0 0 0 0 0 0 0 190 0 0 1 0 0 0 0 0 0 0 0 0 0 191 0 0 0 0 0 0 0 0 1 0 0 0 0 192 0 0 0 0 0 0 0 0 1 0 1 0 0 193 0 0 0 0 0 0 0 0 0 0 0 0 0 194 0 0 0 0 0 0 0 0 1 0 0 0 0 195 0 0 0 0 0 0 0 0 1 0 0 0 0 198 0 0 0 0 0 0 0 0 0 0 0 0 0 199 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 201 0 0 0 0 0 0 0 0 0 0 0 0 0 202 0 0 0 0 0 0 0 0 0 0 0 0 0 203 0 0 0 0 0 0 0 0 0 0 0 0 0 204 0 0 0 0 0 0 0 0 0 0 0 0 0 205 0 0 0 0 0 0 0 0 0 0 0 0 0 218 0 0 0 0 0 0 0 0 0 0 0 0 0 224 0 0 0 0 0 0 0 0 0 0 0 0 0 All 84 66 128 9 136 178 101 18 158 9 152 106 188 CCAvg 2.1 2.2 2.3 2.33 2.4 2.5 2.6 2.67 2.7 2.75 2.8 2.9 3.0 \ Income 8 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0 0 0 0 0 0 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0 38 4 5 0 0 1 2 0 0 0 0 1 0 0 39 6 3 2 0 2 0 1 0 0 0 1 0 0 40 2 4 2 0 3 5 1 0 1 0 1 0 0 41 0 3 1 0 4 6 0 0 0 0 0 0 0 42 4 3 1 0 3 7 0 0 2 0 0 0 0 43 5 0 0 0 1 0 0 0 0 0 2 0 0 44 2 2 2 0 1 1 0 0 0 0 0 0 0 45 1 0 1 0 1 2 1 0 0 0 1 0 0 48 6 5 0 0 1 3 1 0 0 0 2 0 0 49 3 7 1 0 2 5 0 0 0 0 4 0 0 50 5 3 1 1 4 1 0 0 1 0 2 0 0 51 3 2 0 1 0 0 1 0 0 0 2 0 0 52 1 4 0 1 1 3 1 0 1 0 1 0 0 53 3 4 2 0 3 4 1 0 0 0 4 0 0 54 3 4 3 0 1 1 0 0 0 0 1 1 1 55 1 2 1 0 1 4 1 0 0 0 1 0 0 58 2 1 1 0 2 3 1 1 0 0 4 0 0 59 1 1 1 0 5 2 1 2 0 0 1 0 1 60 4 7 1 0 1 4 2 0 0 0 5 0 1 61 2 5 2 0 1 2 4 1 3 0 4 0 1 62 1 4 3 2 2 2 0 2 0 0 1 0 1 63 2 4 1 0 0 2 0 1 0 0 0 0 1 64 2 4 3 0 0 4 3 2 0 0 2 2 1 65 2 8 0 0 3 3 3 1 1 0 4 0 1 68 1 0 3 0 0 0 3 1 0 0 2 2 1 69 4 1 0 2 1 0 0 1 1 0 1 1 0 70 2 2 1 2 0 0 4 1 1 0 3 3 0 71 0 1 1 2 0 1 1 0 1 0 0 2 1 72 0 1 2 0 1 0 5 0 0 0 3 1 0 73 0 2 4 2 4 1 3 2 0 0 1 1 3 74 0 2 2 1 0 1 6 1 0 0 1 0 3 75 1 1 2 4 0 0 2 0 1 0 2 0 1 78 1 1 1 0 4 0 4 4 0 0 2 1 2 79 2 3 0 0 1 2 1 1 2 0 1 1 0 80 0 3 0 0 1 1 2 2 2 0 2 0 1 81 0 1 0 0 1 3 3 3 1 0 3 4 1 82 2 2 0 0 2 4 3 3 1 0 1 2 1 83 1 4 0 0 3 2 1 4 1 0 3 0 2 84 1 4 0 0 2 0 1 1 0 0 2 4 1 85 3 0 0 0 0 5 1 2 1 0 0 0 0 88 0 0 0 0 0 0 1 0 3 0 1 0 0 89 0 0 0 0 0 0 1 0 2 0 2 0 0 90 0 0 0 0 0 0 3 0 4 0 3 0 1 91 0 0 0 0 0 0 2 0 0 0 5 0 1 92 0 0 0 0 0 0 1 0 1 0 3 0 2 93 0 2 0 0 2 0 1 0 5 0 4 0 1 94 0 0 0 0 0 0 1 0 0 0 1 0 0 95 0 0 0 0 0 0 1 0 1 0 0 0 1 98 0 0 1 0 0 1 1 0 0 0 1 0 0 99 0 1 1 0 1 0 0 0 1 0 0 0 1 100 0 0 0 0 0 0 0 0 2 0 0 0 0 101 2 1 1 0 1 1 0 0 1 0 1 1 0 102 1 1 1 0 0 0 1 0 0 0 0 0 1 103 0 0 0 0 0 2 1 0 0 0 1 0 0 104 0 0 0 0 1 3 0 0 0 0 0 0 0 105 0 0 0 0 1 0 0 0 0 0 1 0 2 108 0 0 0 0 1 0 0 0 0 1 0 0 0 109 0 2 1 0 1 1 0 0 0 0 0 0 0 110 0 0 0 0 0 1 0 0 0 0 0 1 0 111 0 0 1 0 0 0 0 0 0 0 0 0 1 112 1 1 1 0 2 2 0 0 1 0 0 0 0 113 1 0 1 0 1 1 0 0 1 0 1 0 0 114 1 0 0 0 3 1 0 0 0 0 0 0 0 115 0 1 0 0 0 1 0 0 1 0 1 1 0 118 0 0 0 0 1 0 0 0 0 0 1 0 1 119 0 0 0 0 0 0 0 0 0 0 0 1 0 120 0 0 0 0 0 0 0 0 1 0 0 0 1 121 1 0 0 0 0 0 0 0 0 0 0 1 0 122 0 0 0 0 1 0 1 0 0 0 0 0 3 123 0 1 1 0 0 0 0 0 0 0 0 2 1 124 0 0 0 0 0 0 0 0 0 0 0 0 0 125 0 1 0 0 4 0 0 0 0 0 0 0 0 128 0 0 0 0 0 2 1 0 1 0 0 0 0 129 0 0 0 0 0 1 0 0 0 0 0 0 0 130 0 0 0 0 0 0 1 0 1 0 0 1 0 131 0 1 0 0 0 0 1 0 3 0 0 1 0 132 0 0 0 0 1 0 1 0 0 0 0 1 0 133 0 0 0 0 0 0 2 0 0 0 0 0 0 134 0 0 0 0 0 0 0 0 0 0 0 0 0 135 0 0 0 0 0 0 0 0 1 0 0 0 0 138 0 1 0 0 0 1 0 0 0 0 1 0 0 139 0 0 0 0 0 0 0 0 0 0 0 1 0 140 0 0 0 0 0 1 0 0 1 0 0 1 0 141 1 0 0 0 2 0 0 0 0 0 0 0 0 142 0 0 1 0 0 0 0 0 1 0 1 0 0 143 0 0 0 0 0 0 0 0 1 0 0 1 0 144 0 0 0 0 0 1 0 0 0 0 0 0 0 145 0 0 0 0 0 1 0 0 1 0 0 1 0 148 0 0 0 0 0 0 0 0 0 0 0 0 0 149 0 0 0 0 0 0 0 0 0 0 0 0 0 150 0 0 0 0 0 0 0 0 0 0 0 0 0 151 0 0 0 0 0 0 0 0 0 0 0 0 0 152 0 0 0 0 0 0 0 0 0 0 0 0 1 153 0 0 0 0 0 0 1 0 0 0 1 0 1 154 0 0 0 0 1 0 0 0 0 0 2 1 1 155 0 0 0 0 0 0 0 0 0 0 0 1 0 158 2 0 1 0 2 0 0 0 0 0 0 0 0 159 0 0 0 0 0 0 0 0 0 0 0 0 0 160 2 0 0 0 0 0 0 0 0 0 0 0 0 161 0 0 0 0 0 0 0 0 0 0 0 3 0 162 0 0 0 0 0 0 0 0 0 0 0 1 0 163 0 0 0 0 1 0 0 0 0 0 0 0 0 164 0 0 0 0 1 0 0 0 0 0 0 0 0 165 0 0 0 0 0 0 0 0 1 0 0 0 0 168 0 0 0 0 0 0 0 0 0 0 1 0 0 169 1 0 0 0 0 0 0 0 0 0 0 0 0 170 1 0 0 0 0 0 0 0 0 0 1 0 0 171 1 1 0 0 0 0 0 0 0 0 0 0 0 172 0 0 0 0 0 0 0 0 0 0 0 0 0 173 0 0 0 0 0 0 0 0 1 0 0 0 0 174 0 0 0 0 1 0 0 0 0 0 0 1 0 175 0 0 0 0 0 0 0 0 0 0 0 0 0 178 0 0 0 0 0 0 0 0 0 0 0 1 0 179 1 0 0 0 0 0 1 0 0 0 0 1 0 180 0 0 0 0 0 0 0 0 0 0 1 2 0 181 0 0 1 0 0 0 0 0 0 0 0 0 0 182 0 0 0 0 0 0 1 0 0 0 0 0 0 183 0 0 0 0 0 0 0 0 0 0 0 0 1 184 0 1 1 0 0 0 0 0 0 0 0 0 0 185 0 1 0 0 0 0 0 0 1 0 0 1 0 188 0 0 0 0 0 0 0 0 0 0 0 1 0 189 0 0 0 0 0 0 0 0 0 0 0 0 0 190 1 1 0 0 0 0 0 0 0 0 0 0 1 191 0 0 0 0 0 0 1 0 0 0 1 1 1 192 0 0 0 0 0 0 0 0 0 0 1 0 0 193 0 0 0 0 0 0 0 0 0 0 0 0 0 194 0 0 0 0 0 0 0 0 0 0 0 0 0 195 0 0 0 0 0 0 0 0 0 0 0 1 3 198 0 0 0 0 0 0 0 0 0 0 1 0 1 199 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 1 201 0 0 0 0 0 0 0 0 0 0 1 0 0 202 0 0 0 0 0 0 0 0 0 0 0 0 0 203 0 0 0 0 0 0 0 0 0 0 0 0 0 204 0 0 0 0 0 0 0 0 0 0 1 0 0 205 0 0 0 0 0 0 0 0 0 0 0 0 0 218 0 0 0 0 0 0 0 0 0 0 0 0 0 224 0 0 0 0 0 0 0 0 0 0 0 0 0 All 100 130 58 18 92 107 87 36 58 1 110 54 53 CCAvg 3.1 3.2 3.25 3.3 3.33 3.4 3.5 3.6 3.67 3.7 3.8 3.9 4.0 \ Income 8 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0 0 0 0 0 0 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0 39 0 0 0 0 0 0 0 0 0 0 0 0 0 40 0 0 0 0 0 0 0 0 0 0 0 0 0 41 0 0 0 0 0 0 0 0 0 0 0 0 0 42 0 0 0 0 0 0 0 0 0 0 0 0 0 43 0 0 0 0 0 0 0 0 0 0 0 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 48 0 0 0 0 0 0 0 0 0 0 0 0 0 49 0 0 0 0 0 0 0 0 0 0 0 0 0 50 0 0 0 0 0 0 0 0 0 0 0 0 0 51 0 2 0 0 0 0 0 0 0 0 0 0 0 52 0 0 0 0 0 0 0 0 0 0 0 0 0 53 0 0 0 0 0 0 0 0 0 0 0 0 0 54 0 0 0 0 0 0 0 0 0 0 0 0 0 55 0 1 0 0 0 0 0 0 0 0 0 0 0 58 0 2 0 0 0 0 0 2 0 0 0 0 0 59 0 0 0 0 0 0 0 1 0 2 0 0 0 60 0 1 0 0 0 0 0 0 0 0 1 0 1 61 0 0 0 0 0 0 0 0 0 0 0 0 0 62 0 1 0 0 0 0 0 1 0 0 1 0 0 63 0 1 0 0 0 0 0 2 0 0 0 0 0 64 0 0 0 0 0 2 0 0 0 0 0 0 0 65 0 2 0 0 0 0 0 0 0 1 0 0 0 68 0 0 0 0 0 0 0 0 0 2 0 0 1 69 1 0 0 0 0 1 0 0 0 0 0 0 2 70 0 0 0 0 0 0 0 0 0 0 0 0 0 71 0 1 0 1 0 0 0 0 0 0 0 0 0 72 0 0 0 0 0 0 0 0 0 1 0 0 0 73 0 0 0 1 0 1 0 0 0 0 1 0 0 74 0 1 0 0 0 0 0 1 0 0 0 0 2 75 0 1 0 0 0 0 1 1 0 2 0 0 0 78 0 0 0 0 0 0 0 2 0 0 1 0 0 79 0 0 0 0 0 0 0 2 0 1 2 0 1 80 0 0 0 0 0 1 0 0 0 0 0 0 0 81 0 2 0 0 0 2 0 2 0 3 3 0 2 82 0 0 0 0 0 2 0 1 0 2 1 0 1 83 2 0 0 0 0 3 0 2 0 0 3 1 1 84 1 0 0 0 0 1 0 1 0 2 0 0 1 85 0 2 0 0 0 4 0 0 0 3 1 0 3 88 0 0 0 0 0 0 0 0 0 0 1 0 0 89 0 0 0 0 0 0 0 0 0 0 1 1 0 90 0 0 0 1 0 0 0 1 0 0 0 0 0 91 0 0 0 0 0 1 0 0 0 0 1 0 0 92 1 0 0 1 0 0 0 0 0 0 0 0 0 93 1 0 0 0 0 0 0 1 0 0 0 0 0 94 1 0 0 0 0 0 0 0 0 0 2 0 0 95 0 0 0 0 0 0 0 0 0 1 1 2 0 98 1 0 0 0 0 0 0 0 0 0 0 1 1 99 0 0 0 0 0 0 1 0 0 0 1 0 1 100 0 0 0 0 0 1 0 0 0 0 0 0 0 101 0 0 0 0 0 1 0 0 0 0 1 2 0 102 0 0 0 1 0 1 0 0 0 0 0 0 0 103 0 0 0 0 1 1 0 0 0 0 0 0 1 104 0 0 0 0 0 1 0 1 0 1 0 0 2 105 0 1 0 1 0 0 0 0 0 0 0 0 2 108 0 0 0 0 0 1 0 0 1 0 2 0 1 109 0 0 0 0 0 0 0 0 0 0 0 0 1 110 0 0 0 1 0 1 0 0 0 0 1 0 1 111 0 0 0 0 0 0 0 1 0 0 1 1 0 112 0 0 0 0 0 0 0 0 0 0 1 0 0 113 0 0 0 2 0 0 0 0 0 0 1 0 0 114 0 0 0 1 0 2 1 0 0 0 1 0 0 115 1 0 0 1 0 2 1 0 0 0 1 0 1 118 0 0 0 3 0 0 0 0 0 0 0 0 0 119 0 0 0 2 0 0 0 0 0 0 0 1 0 120 0 1 0 0 0 0 0 0 0 0 0 1 0 121 1 0 0 2 0 0 1 0 0 0 0 0 0 122 1 0 0 0 0 0 0 0 0 0 0 0 0 123 2 0 0 0 0 0 0 0 0 0 1 0 0 124 0 0 0 0 0 0 0 0 0 0 1 1 0 125 0 0 0 0 0 0 2 0 0 0 0 2 0 128 0 0 1 0 0 1 0 0 0 0 1 2 0 129 0 0 0 2 0 1 0 0 0 0 0 0 0 130 0 1 0 0 0 0 0 0 0 0 0 0 0 131 0 0 0 0 0 1 0 0 0 0 2 0 0 132 0 0 0 0 0 0 0 0 0 0 2 2 0 133 1 0 0 0 0 0 0 0 0 0 2 0 0 134 2 0 0 4 0 0 0 0 0 0 0 1 2 135 0 0 0 2 0 0 0 0 0 0 2 0 0 138 0 0 0 2 0 0 1 0 0 0 0 2 0 139 0 0 0 0 0 2 0 0 0 0 0 1 1 140 0 0 0 0 0 0 1 0 0 0 0 0 1 141 0 0 0 1 0 0 1 0 0 0 0 0 1 142 0 0 0 1 0 1 0 0 0 0 0 2 0 143 0 0 0 0 0 0 1 0 0 0 0 0 0 144 0 0 0 1 0 0 1 0 0 0 0 0 0 145 0 0 0 0 0 0 1 0 0 0 0 0 0 148 0 0 0 1 0 0 0 0 0 0 0 0 0 149 0 0 0 0 0 0 0 0 0 0 0 1 0 150 0 0 0 1 0 0 0 0 0 0 0 0 0 151 0 0 0 1 0 0 1 0 0 0 0 0 0 152 0 0 0 1 0 0 1 1 0 0 0 1 0 153 0 0 0 0 0 0 0 0 0 0 0 0 0 154 0 0 0 0 0 0 0 0 0 0 0 0 0 155 0 0 0 0 0 0 0 0 0 0 0 1 0 158 0 0 0 0 0 0 0 0 0 2 0 0 0 159 0 0 0 0 0 0 0 0 0 0 0 1 0 160 0 0 0 1 0 0 0 0 0 0 1 0 0 161 0 0 0 1 0 1 0 0 0 0 0 0 0 162 0 0 0 1 0 0 0 0 0 0 0 0 0 163 0 0 0 0 0 0 0 0 0 0 0 0 0 164 0 0 0 0 0 0 0 0 0 0 1 0 1 165 0 0 0 1 0 0 0 0 0 0 0 0 0 168 0 0 0 0 0 0 0 0 0 0 0 0 0 169 0 0 0 0 0 0 0 0 0 0 0 0 0 170 0 0 0 0 0 0 0 0 0 0 0 0 0 171 0 0 0 1 0 0 0 0 0 0 0 0 0 172 1 0 0 0 0 1 0 0 0 0 0 0 0 173 1 0 0 1 0 0 0 0 0 0 0 0 0 174 0 1 0 0 0 0 0 0 0 0 0 0 0 175 0 0 0 2 0 0 0 1 0 0 0 0 0 178 0 0 0 0 0 0 0 0 0 0 0 0 0 179 0 0 0 0 0 0 0 1 0 0 0 0 0 180 0 0 0 0 0 0 0 1 0 0 0 0 0 181 0 0 0 1 0 0 0 0 0 0 0 0 0 182 0 1 0 1 0 0 0 0 0 1 0 0 0 183 1 0 0 0 0 1 0 0 0 0 0 0 0 184 0 0 0 0 0 1 0 0 0 0 0 0 0 185 0 0 0 0 0 0 0 1 0 0 0 0 0 188 0 0 0 0 0 0 0 0 0 1 0 0 0 189 0 0 0 0 0 0 0 0 0 0 0 0 0 190 1 0 0 0 0 0 0 0 0 0 0 0 0 191 0 0 0 0 0 0 0 0 0 0 0 0 0 192 0 0 0 0 0 0 0 0 0 0 0 0 0 193 0 0 0 0 0 0 0 0 0 0 0 0 1 194 0 0 0 0 0 0 0 0 0 0 0 0 0 195 0 0 0 0 0 0 0 0 0 0 0 0 0 198 0 0 0 0 0 0 0 0 0 0 0 0 0 199 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 201 0 0 0 0 0 0 0 0 0 0 0 0 0 202 0 0 0 0 0 0 0 0 0 0 0 0 0 203 0 0 0 0 0 0 0 0 0 0 0 0 0 204 0 0 0 0 0 0 0 0 0 0 0 0 0 205 0 0 0 0 0 0 0 0 0 0 0 0 0 218 0 0 0 0 0 0 0 0 0 0 0 0 0 224 0 0 0 0 0 0 0 0 0 0 0 0 0 All 20 22 1 45 1 39 15 27 1 25 43 27 33 CCAvg 4.1 4.2 4.25 4.3 4.33 4.4 4.5 4.6 4.67 4.7 4.75 4.8 4.9 \ Income 8 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0 0 0 0 0 0 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0 39 0 0 0 0 0 0 0 0 0 0 0 0 0 40 0 0 0 0 0 0 0 0 0 0 0 0 0 41 0 0 0 0 0 0 0 0 0 0 0 0 0 42 0 0 0 0 0 0 0 0 0 0 0 0 0 43 0 0 0 0 0 0 0 0 0 0 0 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 48 0 0 0 0 0 0 0 0 0 0 0 0 0 49 0 0 0 0 0 0 0 0 0 0 0 0 0 50 0 0 0 0 0 0 0 0 0 0 0 0 0 51 0 0 0 0 0 0 0 0 0 0 0 0 0 52 0 0 0 0 0 0 0 0 0 0 0 0 0 53 0 0 0 0 0 0 0 0 0 0 0 0 0 54 0 0 0 0 0 0 0 0 0 0 0 0 0 55 0 0 0 0 0 0 0 0 0 0 0 0 0 58 0 0 0 0 0 0 0 0 0 0 0 0 0 59 0 0 0 0 0 0 0 0 0 0 0 0 0 60 0 0 0 0 0 0 0 0 0 0 0 0 0 61 0 0 0 0 0 0 0 0 0 0 0 0 0 62 0 0 0 0 0 0 0 0 0 0 0 0 0 63 0 0 0 0 0 0 0 0 0 0 0 0 0 64 0 0 0 0 0 0 0 0 0 0 0 0 0 65 0 0 0 0 0 0 0 0 0 0 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0 0 0 69 0 0 0 0 0 0 0 0 0 0 0 0 0 70 0 0 0 0 0 0 0 0 0 0 0 0 0 71 0 0 0 0 0 0 0 0 0 0 0 0 0 72 0 0 0 0 0 0 0 0 0 0 0 0 0 73 0 0 0 0 0 0 0 0 0 0 0 0 0 74 0 0 0 0 0 0 0 0 0 0 0 0 0 75 0 0 0 0 0 0 1 0 0 0 0 0 0 78 0 0 0 0 0 1 1 0 0 0 0 0 1 79 0 0 0 0 0 1 0 0 0 0 0 0 1 80 0 0 0 0 0 0 1 0 0 0 0 0 2 81 0 0 0 0 0 1 2 0 0 0 0 0 2 82 0 0 0 1 0 0 0 0 0 0 0 0 0 83 0 0 0 0 0 1 1 0 0 0 0 0 0 84 0 0 0 1 0 0 0 0 0 0 0 0 1 85 0 0 0 0 0 0 0 0 0 0 0 0 2 88 0 0 0 0 0 0 0 0 0 1 0 0 0 89 1 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 3 0 0 0 0 0 0 91 0 0 0 2 0 0 1 0 0 1 0 0 0 92 0 0 0 0 0 0 1 0 0 0 0 0 0 93 1 0 0 1 0 0 0 0 0 0 0 0 1 94 1 0 0 1 0 0 0 0 0 1 0 0 1 95 0 0 0 0 0 0 1 0 0 0 0 0 0 98 0 1 0 0 0 0 1 0 0 0 0 0 0 99 0 0 0 0 0 1 2 1 0 0 0 1 0 100 0 0 0 0 0 0 0 0 0 0 0 0 0 101 0 0 0 0 0 2 0 0 0 0 0 0 1 102 0 0 0 0 0 0 1 0 0 1 0 0 0 103 0 0 0 0 0 0 1 1 0 0 0 1 0 104 0 1 0 0 0 0 0 1 0 0 0 0 0 105 0 0 0 0 0 0 1 1 0 1 0 0 0 108 0 0 0 0 0 1 0 1 0 0 0 0 0 109 0 0 0 0 0 0 0 0 0 0 0 0 0 110 1 0 0 0 0 2 0 0 0 0 0 0 1 111 0 0 0 1 0 1 0 0 0 0 0 0 0 112 0 0 0 0 1 0 0 1 0 0 0 0 0 113 0 0 0 0 0 0 0 0 0 0 0 0 0 114 0 1 0 0 0 1 0 0 0 0 0 0 1 115 0 1 0 0 0 0 0 1 0 0 0 0 0 118 0 0 0 0 0 0 0 0 0 1 0 0 0 119 0 1 0 0 0 0 0 0 0 0 0 0 1 120 1 1 0 0 0 0 0 0 0 0 0 0 0 121 0 0 0 2 0 0 0 0 0 1 0 0 1 122 1 0 0 0 0 0 0 0 0 0 0 0 0 123 0 0 0 0 0 0 0 0 0 0 0 0 0 124 0 0 0 0 0 0 0 0 0 1 0 0 1 125 0 0 0 2 0 0 0 0 0 1 0 0 0 128 1 0 0 0 2 0 0 0 0 2 0 0 0 129 1 1 0 0 0 1 0 0 0 0 0 0 0 130 0 0 0 0 0 1 0 0 0 2 0 0 0 131 0 0 0 0 1 0 0 0 0 0 0 0 1 132 0 0 0 0 0 1 0 0 0 0 0 0 0 133 0 0 0 0 0 0 0 0 0 0 0 0 0 134 0 0 0 0 0 0 1 1 0 0 0 0 0 135 1 0 0 0 0 0 0 1 0 0 0 2 0 138 0 0 0 0 1 0 0 0 1 0 0 0 0 139 0 0 0 2 0 0 0 1 0 0 0 0 0 140 0 0 0 0 0 0 0 1 0 0 0 0 0 141 0 0 0 0 3 0 1 0 0 0 0 0 1 142 0 1 0 0 0 0 0 0 0 0 0 0 0 143 2 0 0 0 0 0 0 0 0 0 0 0 0 144 1 0 0 0 0 0 0 0 0 0 0 0 0 145 0 0 0 1 1 0 0 0 0 0 0 0 0 148 1 0 0 1 0 0 0 0 0 1 0 0 0 149 0 0 0 0 0 0 0 0 0 3 0 0 0 150 0 0 0 0 0 0 0 0 0 0 0 0 0 151 0 0 0 0 0 0 0 0 0 0 0 0 0 152 0 0 0 0 0 0 0 0 0 0 0 0 0 153 0 0 0 0 0 0 0 0 0 0 0 0 0 154 0 0 0 0 0 0 1 1 0 0 0 0 1 155 0 0 0 0 0 0 0 0 0 0 0 0 0 158 0 0 0 0 0 0 0 0 0 1 0 0 0 159 0 0 0 0 0 0 0 0 0 0 0 0 1 160 0 0 0 3 0 0 0 0 0 0 0 0 0 161 2 0 0 1 0 0 0 0 0 0 0 0 0 162 0 0 0 1 0 0 0 0 0 0 0 0 0 163 1 0 0 0 0 0 0 0 0 0 0 0 0 164 0 0 0 1 0 0 0 0 0 0 0 0 0 165 1 0 0 0 0 0 0 0 0 0 0 0 0 168 1 0 0 0 0 0 0 0 0 0 0 0 0 169 0 0 0 0 0 0 0 0 0 0 0 0 0 170 0 0 1 0 0 0 0 0 0 1 0 0 0 171 0 0 0 0 0 0 0 0 0 0 0 0 0 172 0 0 1 1 0 1 1 0 0 0 0 0 0 173 0 0 0 0 0 0 0 1 0 0 0 0 0 174 0 0 0 0 0 0 0 1 0 2 0 0 0 175 0 0 0 0 0 0 0 0 0 0 0 0 0 178 1 0 0 0 0 0 0 0 0 0 0 0 0 179 1 1 0 0 0 0 1 0 0 0 0 0 1 180 1 0 0 1 0 0 0 0 0 0 0 0 0 181 1 0 0 1 0 0 0 0 0 0 0 0 0 182 0 0 0 0 0 0 0 0 0 0 0 0 0 183 0 0 0 0 0 0 0 0 0 0 0 0 0 184 0 1 0 0 0 0 0 0 0 0 0 1 0 185 0 0 0 0 0 0 0 0 0 0 0 0 0 188 0 0 0 0 0 0 2 0 0 0 0 0 0 189 0 0 0 0 0 0 0 0 0 0 1 0 0 190 0 1 0 0 0 0 2 0 0 0 0 0 0 191 0 0 0 1 0 1 0 0 0 0 0 2 0 192 0 0 0 0 0 0 0 0 0 0 0 0 0 193 0 0 0 0 0 0 0 0 0 1 0 0 0 194 0 0 0 0 0 0 1 0 0 0 0 0 0 195 0 0 0 1 0 0 0 0 0 1 1 0 0 198 0 0 0 0 0 0 0 0 0 0 0 0 0 199 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 201 0 0 0 0 0 0 0 0 0 0 0 0 0 202 0 0 0 0 0 0 0 0 0 1 0 0 0 203 0 0 0 0 0 0 0 0 0 0 0 0 0 204 0 0 0 0 0 0 1 0 0 0 0 0 0 205 0 0 0 0 0 0 0 0 0 0 0 0 0 218 0 0 0 0 0 0 0 0 0 0 0 0 0 224 0 0 0 0 0 0 0 0 0 0 0 0 0 All 22 11 2 26 9 17 29 14 1 24 2 7 22 CCAvg 5.0 5.1 5.2 5.3 5.33 5.4 5.5 5.6 5.67 5.7 5.8 5.9 6.0 \ Income 8 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0 0 0 0 0 0 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0 39 0 0 0 0 0 0 0 0 0 0 0 0 0 40 0 0 0 0 0 0 0 0 0 0 0 0 0 41 0 0 0 0 0 0 0 0 0 0 0 0 0 42 0 0 0 0 0 0 0 0 0 0 0 0 0 43 0 0 0 0 0 0 0 0 0 0 0 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 48 0 0 0 0 0 0 0 0 0 0 0 0 0 49 0 0 0 0 0 0 0 0 0 0 0 0 0 50 0 0 0 0 0 0 0 0 0 0 0 0 0 51 0 0 0 0 0 0 0 0 0 0 0 0 0 52 0 0 0 0 0 0 0 0 0 0 0 0 0 53 0 0 0 0 0 0 0 0 0 0 0 0 0 54 0 0 0 0 0 0 0 0 0 0 0 0 0 55 0 0 0 0 0 0 0 0 0 0 0 0 0 58 0 0 0 0 0 0 0 0 0 0 0 0 0 59 0 0 0 0 0 0 0 0 0 0 0 0 0 60 0 0 0 0 0 0 0 0 0 0 0 0 0 61 0 0 0 0 0 0 0 0 0 0 0 0 0 62 0 0 0 0 0 0 0 0 0 0 0 0 0 63 0 0 0 0 0 0 0 0 0 0 0 0 0 64 0 0 0 0 0 0 0 0 0 0 0 0 0 65 0 0 0 0 0 0 0 0 0 0 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0 0 0 69 0 0 0 0 0 0 0 0 0 0 0 0 0 70 0 0 0 0 0 0 0 0 0 0 0 0 0 71 0 0 0 0 0 0 0 0 0 0 0 0 0 72 0 0 0 0 0 0 0 0 0 0 0 0 0 73 0 0 0 0 0 0 0 0 0 0 0 0 0 74 0 0 0 0 0 0 0 0 0 0 0 0 0 75 0 0 0 0 0 0 0 0 0 0 0 0 0 78 0 0 1 0 0 0 0 0 0 0 0 0 0 79 0 0 0 0 0 0 0 0 0 0 0 0 0 80 0 0 1 0 0 0 0 0 0 0 0 0 0 81 0 0 0 0 0 0 0 0 0 0 0 0 0 82 0 0 1 0 0 0 0 0 0 0 0 0 0 83 0 0 0 0 0 0 0 0 0 0 0 0 0 84 0 0 0 0 0 0 0 0 0 0 0 0 0 85 0 0 0 0 0 0 0 0 0 0 0 0 0 88 0 0 0 0 0 1 0 0 0 0 0 0 0 89 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 91 0 0 1 0 0 0 0 0 0 1 0 0 0 92 0 0 0 0 0 0 0 0 0 0 0 0 0 93 0 0 1 0 0 0 0 0 0 0 0 0 0 94 0 0 0 0 0 0 0 0 0 0 0 0 0 95 0 0 0 0 0 0 0 0 0 0 0 0 0 98 0 0 0 0 0 1 0 0 0 0 0 0 0 99 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 0 0 0 0 0 0 0 0 0 0 0 0 101 0 0 0 0 0 0 0 0 0 0 0 0 0 102 0 0 0 0 0 0 0 0 0 0 0 0 0 103 0 0 0 0 0 0 0 0 0 0 0 0 0 104 0 0 0 0 0 0 0 0 0 1 0 0 0 105 0 0 0 0 0 0 0 0 0 0 0 0 0 108 0 0 1 0 0 0 0 0 0 0 0 0 0 109 0 0 0 0 0 0 0 0 0 0 0 0 0 110 0 0 0 0 0 1 0 0 0 0 0 0 0 111 0 0 0 1 0 1 0 0 0 0 0 0 0 112 1 1 0 0 0 0 0 0 0 0 0 0 0 113 0 0 0 0 0 0 0 0 0 0 0 0 0 114 0 0 1 0 0 0 0 0 0 0 0 0 0 115 0 0 0 0 0 0 0 0 0 0 0 0 0 118 0 0 0 0 0 1 0 1 0 1 0 0 1 119 1 0 0 0 0 2 0 0 0 0 0 0 0 120 0 0 1 0 0 1 0 0 0 0 0 0 0 121 0 0 0 0 0 1 0 0 0 0 0 0 0 122 0 1 0 0 0 0 0 0 0 0 0 0 2 123 0 0 1 0 0 1 0 2 0 0 0 1 0 124 2 0 1 0 0 0 0 0 0 0 0 0 0 125 0 0 0 0 0 0 0 0 0 1 0 0 0 128 0 0 0 1 0 0 0 0 0 1 0 0 2 129 1 0 1 0 0 0 0 0 0 1 0 2 2 130 1 0 0 0 0 0 0 0 0 0 0 0 0 131 0 0 0 0 0 1 0 0 0 1 0 0 0 132 1 0 0 0 0 0 0 0 0 0 0 1 1 133 2 0 0 0 0 1 0 0 0 1 0 0 0 134 1 0 0 0 0 0 1 0 0 1 0 0 0 135 0 0 1 1 0 0 0 0 0 0 0 0 0 138 0 0 0 0 0 0 0 0 0 0 0 0 0 139 0 0 0 0 0 0 0 0 0 0 0 0 0 140 0 0 0 0 0 0 0 1 0 0 0 1 0 141 1 1 0 0 0 0 0 0 0 0 0 0 0 142 0 0 0 0 0 0 1 0 0 0 0 0 0 143 0 0 0 0 0 0 0 0 0 0 0 0 0 144 0 0 0 0 0 1 0 0 0 0 0 0 0 145 0 0 0 0 0 0 0 0 0 1 0 0 1 148 0 0 0 0 0 0 0 0 0 0 1 0 0 149 0 0 0 0 0 0 1 0 0 0 0 0 2 150 1 0 0 0 0 1 0 0 0 0 0 0 0 151 0 0 0 0 0 0 0 0 0 0 0 0 1 152 0 0 0 0 0 0 0 0 0 0 0 0 1 153 0 0 0 0 0 0 0 1 0 0 0 0 0 154 2 0 0 0 0 1 0 0 0 0 0 0 1 155 0 0 0 1 0 0 0 0 0 0 0 0 0 158 0 1 0 0 0 1 0 0 0 0 0 0 1 159 0 0 0 0 0 0 1 0 0 0 0 0 0 160 0 0 0 0 0 0 0 0 0 1 0 0 0 161 0 0 0 0 0 0 0 0 0 0 1 0 0 162 0 0 0 0 0 0 0 0 0 0 0 0 0 163 1 0 0 0 1 0 0 0 0 0 0 0 0 164 1 0 0 0 0 0 0 0 0 0 0 0 1 165 1 1 0 0 0 0 0 1 0 0 0 0 1 168 0 0 0 0 0 0 0 0 1 1 0 0 0 169 1 0 1 0 0 0 0 0 1 0 0 0 0 170 0 0 0 0 0 0 0 0 0 0 0 0 1 171 0 0 0 0 0 0 0 0 0 0 0 0 1 172 0 0 1 0 0 0 0 0 0 0 0 0 0 173 0 0 0 0 0 0 0 0 0 0 0 0 1 174 0 0 0 0 0 0 0 0 0 0 0 0 0 175 0 0 0 0 0 0 0 0 0 0 0 0 0 178 0 0 0 0 0 1 0 0 0 0 0 0 1 179 0 0 0 0 0 0 0 0 0 0 0 0 0 180 0 0 0 0 0 0 0 0 0 0 0 0 0 181 0 0 0 0 0 0 0 0 0 0 0 0 1 182 0 0 0 0 0 0 0 0 0 0 1 0 1 183 0 0 0 0 0 0 0 0 0 0 0 0 1 184 0 1 0 0 0 1 0 0 0 0 0 0 1 185 0 0 0 0 0 0 0 0 0 0 0 0 0 188 0 0 0 0 0 0 0 0 0 0 0 0 0 189 0 0 0 0 0 0 0 0 0 0 0 0 0 190 0 0 0 0 0 0 0 0 0 0 0 0 0 191 0 0 1 0 0 0 0 1 0 0 0 0 0 192 0 0 0 0 0 0 0 0 0 0 0 0 0 193 0 0 0 0 0 0 0 0 0 0 0 0 0 194 0 0 0 0 0 0 0 0 0 1 0 0 1 195 0 0 1 0 0 0 0 0 0 0 0 0 0 198 0 0 0 0 0 0 0 0 0 0 0 0 0 199 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 0 201 0 0 0 0 0 0 0 0 0 0 0 0 0 202 0 0 0 0 0 0 0 0 0 0 0 0 0 203 0 0 0 0 0 0 0 0 0 0 0 0 0 204 0 0 0 0 0 0 0 0 0 0 0 0 0 205 0 0 0 0 0 0 0 0 0 0 0 0 0 218 0 0 0 0 0 0 0 0 0 0 0 0 0 224 0 0 0 0 0 0 0 0 0 0 0 0 0 All 18 6 16 4 1 18 4 7 2 13 3 5 26 CCAvg 6.1 6.2 6.3 6.33 6.4 6.5 6.6 6.67 6.7 6.8 6.9 7.0 7.2 \ Income 8 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0 0 0 0 0 0 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0 39 0 0 0 0 0 0 0 0 0 0 0 0 0 40 0 0 0 0 0 0 0 0 0 0 0 0 0 41 0 0 0 0 0 0 0 0 0 0 0 0 0 42 0 0 0 0 0 0 0 0 0 0 0 0 0 43 0 0 0 0 0 0 0 0 0 0 0 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 48 0 0 0 0 0 0 0 0 0 0 0 0 0 49 0 0 0 0 0 0 0 0 0 0 0 0 0 50 0 0 0 0 0 0 0 0 0 0 0 0 0 51 0 0 0 0 0 0 0 0 0 0 0 0 0 52 0 0 0 0 0 0 0 0 0 0 0 0 0 53 0 0 0 0 0 0 0 0 0 0 0 0 0 54 0 0 0 0 0 0 0 0 0 0 0 0 0 55 0 0 0 0 0 0 0 0 0 0 0 0 0 58 0 0 0 0 0 0 0 0 0 0 0 0 0 59 0 0 0 0 0 0 0 0 0 0 0 0 0 60 0 0 0 0 0 0 0 0 0 0 0 0 0 61 0 0 0 0 0 0 0 0 0 0 0 0 0 62 0 0 0 0 0 0 0 0 0 0 0 0 0 63 0 0 0 0 0 0 0 0 0 0 0 0 0 64 0 0 0 0 0 0 0 0 0 0 0 0 0 65 0 0 0 0 0 0 0 0 0 0 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0 0 0 69 0 0 0 0 0 0 0 0 0 0 0 0 0 70 0 0 0 0 0 0 0 0 0 0 0 0 0 71 0 0 0 0 0 0 0 0 0 0 0 0 0 72 0 0 0 0 0 0 0 0 0 0 0 0 0 73 0 0 0 0 0 0 0 0 0 0 0 0 0 74 0 0 0 0 0 0 0 0 0 0 0 0 0 75 0 0 0 0 0 0 0 0 0 0 0 0 0 78 0 0 0 0 0 0 0 0 0 0 0 0 0 79 0 0 0 0 0 0 0 0 0 0 0 0 0 80 0 0 0 0 0 0 0 0 0 0 0 0 0 81 0 0 0 0 0 0 0 0 0 0 0 0 0 82 0 0 0 0 0 0 0 0 0 0 0 0 0 83 0 0 0 0 0 0 0 0 0 0 0 0 0 84 0 0 0 0 0 0 0 0 0 0 0 0 0 85 0 0 0 0 0 0 0 0 0 0 0 0 0 88 0 0 0 0 0 0 0 0 0 0 0 0 0 89 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 91 0 0 0 0 0 0 0 0 0 0 0 0 0 92 0 0 0 0 0 0 0 0 0 0 0 0 0 93 0 0 0 0 0 0 0 0 0 0 0 0 0 94 0 0 0 0 0 0 0 0 0 0 0 0 0 95 0 0 0 0 0 0 0 0 0 0 0 0 0 98 0 0 1 0 0 0 0 0 0 0 0 0 0 99 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 0 1 0 0 0 0 0 0 0 0 0 0 101 0 0 0 0 0 0 0 0 0 0 0 0 0 102 0 0 1 0 0 0 0 0 0 0 0 0 0 103 0 0 0 0 0 0 0 0 0 0 0 0 0 104 0 0 0 0 0 0 0 0 0 0 0 0 0 105 0 0 0 0 0 0 0 0 0 0 0 0 0 108 0 0 0 0 0 0 0 0 0 0 0 0 0 109 0 0 0 0 0 0 0 0 0 1 0 1 0 110 1 0 0 0 0 0 0 0 0 0 0 0 0 111 1 0 1 0 0 0 0 0 0 0 0 0 0 112 0 0 0 0 0 0 0 0 0 1 0 0 0 113 0 0 1 0 0 0 0 0 0 0 0 0 0 114 0 0 0 0 0 0 0 0 0 0 1 0 0 115 0 0 0 0 0 0 0 0 0 0 0 1 0 118 0 0 0 0 0 0 0 0 0 1 0 0 1 119 2 0 0 0 0 0 0 0 0 0 0 0 0 120 1 0 0 0 0 0 0 0 0 0 0 0 0 121 0 0 0 0 0 0 0 0 0 0 0 0 0 122 0 0 0 0 0 0 0 0 0 0 0 1 0 123 0 0 0 0 0 0 0 0 0 0 0 0 0 124 0 0 0 0 0 0 0 0 0 0 0 0 0 125 0 0 0 0 0 0 0 0 0 0 0 0 1 128 0 0 0 0 0 0 0 0 0 0 0 0 0 129 0 0 0 0 0 0 0 0 0 0 0 0 0 130 0 0 2 0 0 1 0 0 1 0 0 0 0 131 0 0 0 0 0 0 0 0 0 0 0 0 0 132 0 0 0 0 0 1 0 0 0 0 0 0 0 133 0 0 0 0 0 0 0 0 0 0 0 0 0 134 0 0 1 0 0 1 0 0 0 0 0 0 0 135 0 0 0 0 0 0 0 0 0 0 0 0 1 138 1 0 0 0 0 0 0 0 0 0 1 2 0 139 0 0 0 0 0 0 0 0 0 0 1 0 0 140 0 0 1 0 0 0 1 0 0 0 1 1 0 141 0 0 1 0 0 0 0 0 0 0 3 0 0 142 0 1 0 0 0 0 0 0 0 0 0 0 0 143 0 0 0 0 1 0 1 0 0 0 0 0 0 144 1 0 0 0 0 0 0 0 0 1 0 0 0 145 1 0 1 0 0 0 0 0 0 2 2 0 0 148 0 0 0 0 0 0 0 0 0 1 0 1 0 149 1 0 0 1 0 0 0 0 0 1 0 0 4 150 0 0 0 1 0 0 0 0 0 0 1 0 1 151 0 0 0 0 0 0 0 0 0 0 0 0 0 152 1 0 0 0 0 1 0 0 0 0 1 0 0 153 0 0 0 0 0 1 0 0 0 0 0 0 0 154 1 0 0 0 1 0 0 0 0 0 1 1 0 155 0 0 0 1 0 1 0 0 0 0 0 3 2 158 1 0 1 0 0 0 0 1 0 0 0 0 0 159 0 0 0 0 0 0 0 0 0 0 0 0 0 160 0 0 0 0 0 0 0 1 0 0 0 0 0 161 0 0 0 0 0 1 0 0 0 0 0 0 1 162 0 0 0 0 0 0 0 0 0 0 0 0 0 163 0 0 0 0 0 0 0 0 0 0 0 0 0 164 0 0 0 0 0 0 0 0 0 0 0 0 0 165 0 0 0 0 0 0 0 0 0 0 0 1 0 168 0 0 0 0 0 1 0 0 0 0 0 0 0 169 1 0 0 0 0 1 0 0 0 1 0 0 0 170 1 1 0 0 0 1 0 0 1 0 0 0 0 171 0 0 0 0 0 0 0 0 0 0 0 0 0 172 0 0 0 0 0 1 0 1 0 0 1 0 0 173 0 0 0 0 0 0 1 0 1 0 0 0 1 174 0 0 0 0 0 0 0 0 1 1 0 0 0 175 0 0 0 0 0 0 0 0 2 0 0 0 0 178 0 0 0 0 0 1 0 0 1 0 0 0 0 179 0 0 0 0 0 0 1 0 0 0 0 0 0 180 0 0 0 0 0 2 0 0 1 0 0 0 0 181 0 0 0 0 0 0 0 0 0 0 0 0 0 182 0 0 0 0 0 0 0 0 0 0 0 0 1 183 0 0 0 0 0 0 0 0 0 0 0 0 0 184 0 0 0 0 0 0 0 0 0 0 0 0 0 185 0 0 0 0 0 0 0 0 1 0 0 0 0 188 0 0 0 0 0 1 0 0 0 0 1 1 0 189 0 0 0 0 0 0 0 0 0 0 0 0 0 190 0 0 0 0 0 0 0 0 0 0 0 0 0 191 0 0 0 1 0 0 0 0 0 0 0 0 0 192 0 0 0 0 1 0 0 0 0 0 0 1 0 193 0 0 1 0 0 1 0 0 0 0 0 0 0 194 0 0 0 0 0 0 0 0 0 0 0 0 0 195 0 0 0 3 0 1 0 0 0 0 0 0 0 198 0 0 0 0 0 0 0 1 0 0 0 0 0 199 0 0 0 1 0 0 0 2 0 0 0 0 0 200 0 0 0 0 0 1 0 0 0 0 0 0 0 201 0 0 0 1 0 0 0 1 0 0 0 0 0 202 0 0 0 0 0 0 0 0 0 0 0 0 0 203 0 0 0 0 0 0 0 0 0 0 0 0 0 204 0 0 0 0 0 0 0 0 0 0 0 0 0 205 0 0 0 1 0 0 0 0 0 0 0 0 0 218 0 0 0 0 0 0 0 1 0 0 0 0 0 224 0 0 0 0 0 0 0 1 0 0 0 0 0 All 14 2 13 10 3 18 4 9 9 10 14 14 13 CCAvg 7.3 7.4 7.5 7.6 7.8 7.9 8.0 8.1 8.2 8.3 8.5 8.6 8.8 8.9 \ Income 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 39 0 0 0 0 0 0 0 0 0 0 0 0 0 0 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 42 0 0 0 0 0 0 0 0 0 0 0 0 0 0 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 49 0 0 0 0 0 0 0 0 0 0 0 0 0 0 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 60 0 0 0 0 0 0 0 0 0 0 0 0 0 0 61 0 0 0 0 0 0 0 0 0 0 0 0 0 0 62 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 70 0 0 0 0 0 0 0 0 0 0 0 0 0 0 71 0 0 0 0 0 0 0 0 0 0 0 0 0 0 72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 75 0 0 0 0 0 0 0 0 0 0 0 0 0 0 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 82 0 0 0 0 0 0 0 0 0 0 0 0 0 0 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 84 0 0 0 0 0 0 0 0 0 0 0 0 0 0 85 0 0 0 0 0 0 0 0 0 0 0 0 0 0 88 0 0 0 0 0 0 0 0 0 0 0 0 0 0 89 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 92 0 0 0 0 0 0 0 0 0 0 0 0 0 0 93 0 0 0 0 0 0 0 0 0 0 0 0 0 0 94 0 0 0 0 0 0 0 0 0 0 0 0 0 0 95 0 0 0 0 0 0 0 0 0 0 0 0 0 0 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 99 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 101 0 0 0 0 0 0 0 0 0 0 0 0 0 0 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 103 0 0 0 0 0 0 0 0 0 0 0 0 0 0 104 0 0 0 0 0 0 0 0 0 0 0 0 0 0 105 0 0 0 0 0 0 0 0 0 0 0 0 0 0 108 0 0 0 0 0 0 0 0 0 0 0 0 0 0 109 0 0 0 0 0 0 0 0 0 0 0 0 0 0 110 0 0 0 0 0 0 0 0 0 0 0 0 0 0 111 0 0 0 0 0 0 0 0 0 0 0 0 0 0 112 0 0 0 0 0 0 0 0 0 0 0 0 0 0 113 0 0 0 0 0 0 0 0 0 0 0 0 0 0 114 0 0 0 0 0 0 0 0 0 0 0 0 0 0 115 0 0 0 0 0 0 0 0 0 0 0 0 0 0 118 0 0 1 0 1 0 0 0 0 0 0 0 0 0 119 1 0 1 0 1 0 0 0 0 0 0 0 0 0 120 0 2 1 0 0 0 0 0 0 0 0 0 0 0 121 1 0 0 0 0 0 0 0 0 0 0 0 0 0 122 0 0 0 0 0 0 1 0 0 0 0 0 0 0 123 1 0 0 0 0 0 0 0 0 0 0 0 0 0 124 0 1 0 0 0 0 0 0 0 0 0 0 0 0 125 1 0 0 0 0 0 0 0 0 0 0 0 0 0 128 0 1 0 0 0 0 0 0 0 0 0 0 0 0 129 0 1 0 0 0 0 0 0 0 0 0 0 0 0 130 0 0 1 0 0 0 0 0 0 0 0 0 0 0 131 0 0 0 0 0 0 0 0 0 0 0 0 0 0 132 0 0 0 0 0 0 0 0 0 0 0 0 0 0 133 1 0 0 0 0 0 0 0 0 0 0 0 0 0 134 0 0 0 0 0 0 0 0 0 0 0 0 0 0 135 0 0 0 0 0 0 0 0 0 0 0 0 0 0 138 0 0 0 0 1 0 0 0 0 0 0 0 0 0 139 0 0 0 0 1 0 1 0 0 0 0 0 0 0 140 0 0 0 1 0 0 0 0 0 0 0 0 0 0 141 0 0 0 1 0 0 1 0 0 0 0 0 0 0 142 0 0 0 0 0 0 0 0 0 0 0 0 0 0 143 0 0 0 0 0 0 0 0 0 0 0 1 0 0 144 0 0 0 0 0 0 0 0 0 0 0 0 0 0 145 0 0 0 0 0 0 1 1 0 0 0 0 0 0 148 0 0 1 0 0 0 0 0 0 0 0 0 0 0 149 0 0 0 0 0 0 0 0 0 0 0 0 0 0 150 2 1 0 0 0 0 0 0 0 0 0 0 0 0 151 0 0 0 0 0 0 0 0 0 0 0 0 0 0 152 1 1 1 0 0 0 0 0 0 0 0 0 0 0 153 0 0 2 0 0 0 0 0 0 0 0 0 0 0 154 0 1 1 0 0 0 0 0 0 0 0 0 1 0 155 1 2 0 0 0 0 0 0 0 0 0 0 0 0 158 0 1 0 0 1 0 0 0 0 0 0 0 0 0 159 0 0 0 0 0 0 0 0 0 0 0 0 0 0 160 0 0 0 1 0 0 2 0 0 0 0 0 0 0 161 0 0 1 1 0 1 1 0 0 0 0 0 0 0 162 0 0 0 0 0 0 0 1 0 0 0 2 0 0 163 0 1 0 0 0 0 0 0 0 0 0 0 0 0 164 0 1 0 1 2 0 0 0 0 0 0 0 0 0 165 0 0 0 1 0 0 1 0 0 0 0 0 0 0 168 0 0 0 0 0 1 1 0 0 0 0 0 0 0 169 0 0 0 0 0 0 0 0 0 0 0 0 0 0 170 0 0 0 0 0 1 0 0 0 0 0 0 0 0 171 0 0 0 0 1 0 1 0 0 0 0 0 0 0 172 0 0 0 0 0 0 0 0 0 0 0 0 0 0 173 0 0 0 0 0 0 0 0 0 0 0 0 0 0 174 0 0 0 0 0 0 0 0 0 0 0 0 0 0 175 0 0 0 0 1 0 1 0 0 0 0 0 0 0 178 0 0 0 0 0 0 0 0 0 0 1 0 0 0 179 0 0 0 0 0 0 0 1 0 0 0 2 0 0 180 0 0 0 1 0 0 0 0 0 0 1 1 0 1 181 0 0 0 0 0 0 0 1 0 0 0 0 0 0 182 0 0 0 1 0 0 0 0 0 0 0 1 0 0 183 0 0 0 0 0 0 0 1 1 1 0 0 1 0 184 0 0 1 0 0 0 1 2 0 0 0 0 0 0 185 0 0 1 0 0 1 0 0 0 0 0 0 0 0 188 0 0 0 0 0 0 0 0 0 0 0 0 0 0 189 0 0 0 1 0 0 0 0 0 0 0 0 0 0 190 1 0 0 0 0 0 0 0 0 0 0 0 1 0 191 0 0 0 0 0 0 0 1 0 0 0 0 0 0 192 0 0 0 0 0 0 0 0 0 0 0 0 0 0 193 0 0 0 0 0 0 0 1 0 0 0 1 0 0 194 0 0 0 0 0 0 0 0 0 1 0 0 1 0 195 0 0 0 0 0 0 0 1 0 0 0 0 0 0 198 0 0 0 0 0 0 0 0 0 0 0 0 0 0 199 0 0 0 0 0 0 0 0 0 0 0 0 0 0 200 0 0 0 0 0 0 0 0 0 0 0 0 1 0 201 0 0 0 0 0 0 0 0 0 0 0 0 1 0 202 0 0 0 0 0 0 0 0 0 0 0 0 0 0 203 0 0 0 0 0 0 0 0 0 0 0 0 1 0 204 0 0 0 0 0 0 0 0 0 0 0 0 1 0 205 0 0 0 0 0 0 0 0 0 0 0 0 1 0 218 0 0 0 0 0 0 0 0 0 0 0 0 0 0 224 0 0 0 0 0 0 0 0 0 0 0 0 0 0 All 10 13 12 9 9 4 12 10 1 2 2 8 9 1 CCAvg 9.0 9.3 10.0 All Income 8 0 0 0 23 9 0 0 0 26 10 0 0 0 23 11 0 0 0 27 12 0 0 0 30 13 0 0 0 32 14 0 0 0 31 15 0 0 0 33 18 0 0 0 53 19 0 0 0 52 20 0 0 0 47 21 0 0 0 65 22 0 0 0 65 23 0 0 0 54 24 0 0 0 47 25 0 0 0 64 28 0 0 0 63 29 0 0 0 67 30 0 0 0 63 31 0 0 0 55 32 0 0 0 58 33 0 0 0 51 34 0 0 0 53 35 0 0 0 65 38 0 0 0 84 39 0 0 0 81 40 0 0 0 78 41 0 0 0 82 42 0 0 0 77 43 0 0 0 70 44 0 0 0 85 45 0 0 0 69 48 0 0 0 44 49 0 0 0 52 50 0 0 0 45 51 0 0 0 41 52 0 0 0 47 53 0 0 0 57 54 0 0 0 52 55 0 0 0 61 58 0 0 0 55 59 0 0 0 53 60 0 0 0 52 61 0 0 0 57 62 0 0 0 55 63 0 0 0 46 64 0 0 0 60 65 0 0 0 60 68 0 0 0 35 69 0 0 0 46 70 0 0 0 47 71 0 0 0 43 72 0 0 0 41 73 0 0 0 44 74 0 0 0 45 75 0 0 0 47 78 0 0 0 61 79 0 0 0 53 80 0 0 0 56 81 0 0 0 83 82 0 0 0 61 83 0 0 0 74 84 0 0 0 63 85 0 0 0 65 88 0 0 0 26 89 0 0 0 34 90 0 0 0 38 91 0 0 0 37 92 0 0 0 29 93 0 0 0 37 94 0 0 0 26 95 0 0 0 25 98 0 0 0 28 99 0 0 0 24 100 0 0 0 10 101 0 0 0 24 102 0 0 0 16 103 0 0 0 18 104 0 0 0 20 105 0 0 0 20 108 0 0 0 16 109 0 0 0 18 110 0 0 0 19 111 0 0 0 22 112 0 0 0 26 113 0 0 0 34 114 0 0 0 30 115 0 0 0 27 118 0 0 0 19 119 0 0 0 18 120 0 0 0 17 121 0 0 0 20 122 0 0 0 24 123 0 0 0 18 124 0 0 0 12 125 0 0 0 23 128 0 0 0 24 129 0 0 0 23 130 0 0 0 19 131 0 0 0 19 132 0 0 0 18 133 0 0 0 15 134 0 0 0 20 135 0 0 0 18 138 0 0 0 18 139 0 0 0 16 140 0 0 0 19 141 0 0 0 24 142 0 0 0 15 143 0 0 0 9 144 0 0 0 7 145 0 0 0 23 148 0 0 0 11 149 0 0 0 20 150 0 0 0 11 151 0 0 0 4 152 0 0 0 15 153 0 0 0 11 154 0 0 0 21 155 0 0 0 19 158 0 0 0 18 159 0 0 0 7 160 0 0 0 12 161 0 0 0 16 162 0 0 0 10 163 0 0 0 9 164 0 0 0 13 165 0 0 0 11 168 0 0 0 8 169 0 0 0 7 170 0 0 0 12 171 0 0 0 9 172 0 0 0 11 173 0 0 0 13 174 0 0 0 9 175 0 0 0 12 178 1 0 0 10 179 0 0 0 17 180 1 0 0 18 181 0 0 0 8 182 0 0 0 13 183 0 0 0 12 184 0 0 0 12 185 0 0 0 9 188 0 1 0 10 189 0 0 0 2 190 0 0 0 11 191 0 0 0 13 192 0 0 0 6 193 0 0 0 6 194 0 0 0 8 195 0 0 0 15 198 0 0 0 3 199 0 0 0 3 200 0 0 0 3 201 0 0 1 5 202 0 0 1 2 203 0 0 1 2 204 0 0 0 3 205 0 0 0 2 218 0 0 0 1 224 0 0 0 1 All 2 1 3 5000 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
data = data.drop(['Income','ID'],axis=1)
dummy_data = pd.get_dummies(data, columns=[ 'Family','Age',
'Education', 'Securities_Account',
'CD_Account', 'Online', 'CreditCard'],drop_first=True)
dummy_data.head()
| Experience | ZIPCode | CCAvg | Mortgage | Personal_Loan | Family_2 | Family_3 | Family_4 | Age_24 | Age_25 | Age_26 | Age_27 | Age_28 | Age_29 | Age_30 | Age_31 | Age_32 | Age_33 | Age_34 | Age_35 | Age_36 | Age_37 | Age_38 | Age_39 | Age_40 | Age_41 | Age_42 | Age_43 | Age_44 | Age_45 | Age_46 | Age_47 | Age_48 | Age_49 | Age_50 | Age_51 | Age_52 | Age_53 | Age_54 | Age_55 | Age_56 | Age_57 | Age_58 | Age_59 | Age_60 | Age_61 | Age_62 | Age_63 | Age_64 | Age_65 | Age_66 | Age_67 | Education_2 | Education_3 | Securities_Account_1 | CD_Account_1 | Online_1 | CreditCard_1 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 91107 | 1.6 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
| 1 | 19 | 90089 | 1.5 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
| 2 | 15 | 94720 | 1.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 9 | 94112 | 2.7 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
| 4 | 8 | 91330 | 1.0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 |
column_names = list(dummy_data.columns)
column_names.remove('Personal_Loan')
feature_names = column_names
print(feature_names)
['Experience', 'ZIPCode', 'CCAvg', 'Mortgage', 'Family_2', 'Family_3', 'Family_4', 'Age_24', 'Age_25', 'Age_26', 'Age_27', 'Age_28', 'Age_29', 'Age_30', 'Age_31', 'Age_32', 'Age_33', 'Age_34', 'Age_35', 'Age_36', 'Age_37', 'Age_38', 'Age_39', 'Age_40', 'Age_41', 'Age_42', 'Age_43', 'Age_44', 'Age_45', 'Age_46', 'Age_47', 'Age_48', 'Age_49', 'Age_50', 'Age_51', 'Age_52', 'Age_53', 'Age_54', 'Age_55', 'Age_56', 'Age_57', 'Age_58', 'Age_59', 'Age_60', 'Age_61', 'Age_62', 'Age_63', 'Age_64', 'Age_65', 'Age_66', 'Age_67', 'Education_2', 'Education_3', 'Securities_Account_1', 'CD_Account_1', 'Online_1', 'CreditCard_1']
Since we want to explore ways of converting its liability customers to personal loan customers (while retaining them as depositors). we should use Recall as a metric of model evaluation instead of accuracy.
Recall - It gives the ratio of True positives to Actual positives, so high Recall implies low false negatives, i.e. low chances of predicting whether a liability customer will buy a personal loan or not incorrectly.
X = dummy_data.drop('Personal_Loan',axis=1)
y = dummy_data['Personal_Loan'].astype('int64')
X_train, X_test, y_train, y_test =train_test_split(X, y, test_size=0.3, random_state=1)
print(X_train.shape, X_test.shape)
(3500, 57) (1500, 57)
print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)
(3500, 57) (1500, 57) (3500,) (1500,)
model = DecisionTreeClassifier(criterion='gini',class_weight={0:0.10,1:0.90},random_state=1)
model.fit(X_train, y_train)
DecisionTreeClassifier(class_weight={0: 0.1, 1: 0.9}, random_state=1)
def make_confusion_matrix(model,y_actual,labels=[1, 0]):
'''
model : classifier to predict values of X
y_actual : ground truth
'''
y_predict = model.predict(X_test)
cm=metrics.confusion_matrix( y_actual, y_predict, labels=[0, 1])
df_cm = pd.DataFrame(cm, index = [i for i in ["Actual - No","Actual - Yes"]],
columns = [i for i in ['Predicted - No','Predicted - Yes']])
group_counts = ["{0:0.0f}".format(value) for value in
cm.flatten()]
group_percentages = ["{0:.2%}".format(value) for value in
cm.flatten()/np.sum(cm)]
labels = [f"{v1}\n{v2}" for v1, v2 in
zip(group_counts,group_percentages)]
labels = np.asarray(labels).reshape(2,2)
plt.figure(figsize = (10,7))
sns.heatmap(df_cm, annot=labels,fmt='')
plt.ylabel('True label')
plt.xlabel('Predicted label')
make_confusion_matrix(model,y_test)
X_train.head()
| Experience | ZIPCode | CCAvg | Mortgage | Family_2 | Family_3 | Family_4 | Age_24 | Age_25 | Age_26 | Age_27 | Age_28 | Age_29 | Age_30 | Age_31 | Age_32 | Age_33 | Age_34 | Age_35 | Age_36 | Age_37 | Age_38 | Age_39 | Age_40 | Age_41 | Age_42 | Age_43 | Age_44 | Age_45 | Age_46 | Age_47 | Age_48 | Age_49 | Age_50 | Age_51 | Age_52 | Age_53 | Age_54 | Age_55 | Age_56 | Age_57 | Age_58 | Age_59 | Age_60 | Age_61 | Age_62 | Age_63 | Age_64 | Age_65 | Age_66 | Age_67 | Education_2 | Education_3 | Securities_Account_1 | CD_Account_1 | Online_1 | CreditCard_1 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1334 | 22 | 94304 | 1.3 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
| 4768 | 14 | 93118 | 2.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
| 65 | 35 | 91360 | 3.8 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
| 177 | 3 | 94132 | 1.8 | 244 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
| 4489 | 13 | 95518 | 0.2 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
y_train.value_counts(1)
0 0.905429 1 0.094571 Name: Personal_Loan, dtype: float64
## Function to calculate recall score
def get_recall_score(model):
'''
model : classifier to predict values of X
'''
pred_train = model.predict(X_train)
pred_test = model.predict(X_test)
print("Recall on training set : ",metrics.recall_score(y_train,pred_train))
print("Recall on test set : ",metrics.recall_score(y_test,pred_test))
get_recall_score(model)
Recall on training set : 1.0 Recall on test set : 0.5100671140939598
plt.figure(figsize=(20,30))
out = tree.plot_tree(model,feature_names=feature_names,filled=True,fontsize=9,node_ids=False,class_names=None,)
#below code will add arrows to the decision tree split if they are missing
for o in out:
arrow = o.arrow_patch
if arrow is not None:
arrow.set_edgecolor('black')
arrow.set_linewidth(1)
plt.show()
print(tree.export_text(model,feature_names=feature_names,show_weights=True))
|--- CCAvg <= 2.85 | |--- CD_Account_1 <= 0.50 | | |--- Mortgage <= 281.00 | | | |--- CCAvg <= 1.05 | | | | |--- ZIPCode <= 90454.00 | | | | | |--- Age_57 <= 0.50 | | | | | | |--- Age_26 <= 0.50 | | | | | | | |--- Age_31 <= 0.50 | | | | | | | | |--- Age_53 <= 0.50 | | | | | | | | | |--- weights: [12.30, 0.00] class: 0 | | | | | | | | |--- Age_53 > 0.50 | | | | | | | | | |--- Education_3 <= 0.50 | | | | | | | | | | |--- ZIPCode <= 90047.00 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | | |--- ZIPCode > 90047.00 | | | | | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | | | | |--- Education_3 > 0.50 | | | | | | | | | | |--- ZIPCode <= 90154.50 | | | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | | | | |--- ZIPCode > 90154.50 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | |--- Age_31 > 0.50 | | | | | | | | |--- Securities_Account_1 <= 0.50 | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | | |--- Securities_Account_1 > 0.50 | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | |--- Age_26 > 0.50 | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | |--- Age_57 > 0.50 | | | | | | |--- CreditCard_1 <= 0.50 | | | | | | | |--- Family_3 <= 0.50 | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | |--- Family_3 > 0.50 | | | | | | | | |--- Experience <= 31.50 | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | | |--- Experience > 31.50 | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | |--- CreditCard_1 > 0.50 | | | | | | | |--- ZIPCode <= 90163.00 | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | |--- ZIPCode > 90163.00 | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | |--- ZIPCode > 90454.00 | | | | | |--- Age_30 <= 0.50 | | | | | | |--- Age_35 <= 0.50 | | | | | | | |--- Age_38 <= 0.50 | | | | | | | | |--- Age_53 <= 0.50 | | | | | | | | | |--- weights: [99.70, 0.00] class: 0 | | | | | | | | |--- Age_53 > 0.50 | | | | | | | | | |--- ZIPCode <= 94859.00 | | | | | | | | | | |--- weights: [2.70, 0.00] class: 0 | | | | | | | | | |--- ZIPCode > 94859.00 | | | | | | | | | | |--- ZIPCode <= 95000.50 | | | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | | | | |--- ZIPCode > 95000.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | |--- Age_38 > 0.50 | | | | | | | | |--- ZIPCode <= 92065.00 | | | | | | | | | |--- ZIPCode <= 92032.50 | | | | | | | | | | |--- Mortgage <= 98.00 | | | | | | | | | | | |--- weights: [0.40, 0.00] class: 0 | | | | | | | | | | |--- Mortgage > 98.00 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- ZIPCode > 92032.50 | | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | | |--- ZIPCode > 92065.00 | | | | | | | | | |--- weights: [2.20, 0.00] class: 0 | | | | | | |--- Age_35 > 0.50 | | | | | | | |--- Education_3 <= 0.50 | | | | | | | | |--- Securities_Account_1 <= 0.50 | | | | | | | | | |--- weights: [1.90, 0.00] class: 0 | | | | | | | | |--- Securities_Account_1 > 0.50 | | | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | | |--- Education_3 > 0.50 | | | | | | | | |--- CCAvg <= 0.75 | | | | | | | | | |--- CreditCard_1 <= 0.50 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- CreditCard_1 > 0.50 | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | |--- CCAvg > 0.75 | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | |--- Age_30 > 0.50 | | | | | | |--- Education_3 <= 0.50 | | | | | | | |--- weights: [2.20, 0.00] class: 0 | | | | | | |--- Education_3 > 0.50 | | | | | | | |--- ZIPCode <= 93932.50 | | | | | | | | |--- ZIPCode <= 91699.00 | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | |--- ZIPCode > 91699.00 | | | | | | | | | |--- CCAvg <= 0.35 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- CCAvg > 0.35 | | | | | | | | | | |--- weights: [0.00, 1.80] class: 1 | | | | | | | |--- ZIPCode > 93932.50 | | | | | | | | |--- Family_4 <= 0.50 | | | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | | | |--- Family_4 > 0.50 | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | |--- CCAvg > 1.05 | | | | |--- Age_26 <= 0.50 | | | | | |--- Education_3 <= 0.50 | | | | | | |--- Age_43 <= 0.50 | | | | | | | |--- Age_28 <= 0.50 | | | | | | | | |--- ZIPCode <= 94085.50 | | | | | | | | | |--- Age_50 <= 0.50 | | | | | | | | | | |--- Age_45 <= 0.50 | | | | | | | | | | | |--- truncated branch of depth 4 | | | | | | | | | | |--- Age_45 > 0.50 | | | | | | | | | | | |--- truncated branch of depth 4 | | | | | | | | | |--- Age_50 > 0.50 | | | | | | | | | | |--- Family_3 <= 0.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | | | |--- Family_3 > 0.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | |--- ZIPCode > 94085.50 | | | | | | | | | |--- ZIPCode <= 94094.00 | | | | | | | | | | |--- Experience <= 20.50 | | | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | | | | |--- Experience > 20.50 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- ZIPCode > 94094.00 | | | | | | | | | | |--- Mortgage <= 37.50 | | | | | | | | | | | |--- truncated branch of depth 10 | | | | | | | | | | |--- Mortgage > 37.50 | | | | | | | | | | | |--- weights: [8.40, 0.00] class: 0 | | | | | | | |--- Age_28 > 0.50 | | | | | | | | |--- ZIPCode <= 92005.50 | | | | | | | | | |--- Family_4 <= 0.50 | | | | | | | | | | |--- Mortgage <= 191.00 | | | | | | | | | | | |--- weights: [0.00, 1.80] class: 1 | | | | | | | | | | |--- Mortgage > 191.00 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- Family_4 > 0.50 | | | | | | | | | | |--- Education_2 <= 0.50 | | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | | | |--- Education_2 > 0.50 | | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | |--- ZIPCode > 92005.50 | | | | | | | | | |--- weights: [1.60, 0.00] class: 0 | | | | | | |--- Age_43 > 0.50 | | | | | | | |--- Experience <= 18.50 | | | | | | | | |--- Experience <= 16.50 | | | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | | | |--- Experience > 16.50 | | | | | | | | | |--- weights: [1.20, 0.00] class: 0 | | | | | | | |--- Experience > 18.50 | | | | | | | | |--- Family_3 <= 0.50 | | | | | | | | | |--- CreditCard_1 <= 0.50 | | | | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | | | | |--- CreditCard_1 > 0.50 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | |--- Family_3 > 0.50 | | | | | | | | | |--- ZIPCode <= 90601.00 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- ZIPCode > 90601.00 | | | | | | | | | | |--- weights: [0.00, 1.80] class: 1 | | | | | |--- Education_3 > 0.50 | | | | | | |--- CCAvg <= 2.35 | | | | | | | |--- CCAvg <= 1.35 | | | | | | | | |--- Family_2 <= 0.50 | | | | | | | | | |--- weights: [3.90, 0.00] class: 0 | | | | | | | | |--- Family_2 > 0.50 | | | | | | | | | |--- Age_38 <= 0.50 | | | | | | | | | | |--- weights: [0.00, 3.60] class: 1 | | | | | | | | | |--- Age_38 > 0.50 | | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | |--- CCAvg > 1.35 | | | | | | | | |--- Age_60 <= 0.50 | | | | | | | | | |--- Age_50 <= 0.50 | | | | | | | | | | |--- ZIPCode <= 95052.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | | | |--- ZIPCode > 95052.50 | | | | | | | | | | | |--- truncated branch of depth 5 | | | | | | | | | |--- Age_50 > 0.50 | | | | | | | | | | |--- Mortgage <= 188.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | | | |--- Mortgage > 188.50 | | | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | | |--- Age_60 > 0.50 | | | | | | | | | |--- CreditCard_1 <= 0.50 | | | | | | | | | | |--- Securities_Account_1 <= 0.50 | | | | | | | | | | | |--- weights: [0.40, 0.00] class: 0 | | | | | | | | | | |--- Securities_Account_1 > 0.50 | | | | | | | | | | | |--- weights: [0.40, 0.00] class: 0 | | | | | | | | | |--- CreditCard_1 > 0.50 | | | | | | | | | | |--- Experience <= 34.50 | | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | | | |--- Experience > 34.50 | | | | | | | | | | | |--- weights: [0.00, 1.80] class: 1 | | | | | | |--- CCAvg > 2.35 | | | | | | | |--- Experience <= 33.00 | | | | | | | | |--- ZIPCode <= 93508.00 | | | | | | | | | |--- Age_39 <= 0.50 | | | | | | | | | | |--- Family_4 <= 0.50 | | | | | | | | | | | |--- truncated branch of depth 3 | | | | | | | | | | |--- Family_4 > 0.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | | |--- Age_39 > 0.50 | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | |--- ZIPCode > 93508.00 | | | | | | | | | |--- Family_2 <= 0.50 | | | | | | | | | | |--- Age_39 <= 0.50 | | | | | | | | | | | |--- weights: [1.70, 0.00] class: 0 | | | | | | | | | | |--- Age_39 > 0.50 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- Family_2 > 0.50 | | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | |--- Experience > 33.00 | | | | | | | | |--- weights: [1.50, 0.00] class: 0 | | | | |--- Age_26 > 0.50 | | | | | |--- ZIPCode <= 90846.50 | | | | | | |--- weights: [0.50, 0.00] class: 0 | | | | | |--- ZIPCode > 90846.50 | | | | | | |--- ZIPCode <= 94810.50 | | | | | | | |--- CCAvg <= 1.90 | | | | | | | | |--- Mortgage <= 186.00 | | | | | | | | | |--- Education_2 <= 0.50 | | | | | | | | | | |--- weights: [0.40, 0.00] class: 0 | | | | | | | | | |--- Education_2 > 0.50 | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | |--- Mortgage > 186.00 | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | |--- CCAvg > 1.90 | | | | | | | | |--- weights: [0.00, 1.80] class: 1 | | | | | | |--- ZIPCode > 94810.50 | | | | | | | |--- Online_1 <= 0.50 | | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | | |--- Online_1 > 0.50 | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | |--- Mortgage > 281.00 | | | |--- Family_2 <= 0.50 | | | | |--- ZIPCode <= 95025.50 | | | | | |--- Age_63 <= 0.50 | | | | | | |--- Mortgage <= 346.50 | | | | | | | |--- Mortgage <= 327.50 | | | | | | | | |--- Experience <= 32.00 | | | | | | | | | |--- CreditCard_1 <= 0.50 | | | | | | | | | | |--- Experience <= 0.50 | | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | | | |--- Experience > 0.50 | | | | | | | | | | | |--- truncated branch of depth 10 | | | | | | | | | |--- CreditCard_1 > 0.50 | | | | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | | | |--- Experience > 32.00 | | | | | | | | | |--- weights: [0.40, 0.00] class: 0 | | | | | | | |--- Mortgage > 327.50 | | | | | | | | |--- weights: [0.60, 0.00] class: 0 | | | | | | |--- Mortgage > 346.50 | | | | | | | |--- Age_55 <= 0.50 | | | | | | | | |--- Age_53 <= 0.50 | | | | | | | | | |--- Experience <= 2.00 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- Experience > 2.00 | | | | | | | | | | |--- ZIPCode <= 92944.00 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | | | |--- ZIPCode > 92944.00 | | | | | | | | | | | |--- weights: [0.00, 3.60] class: 1 | | | | | | | | |--- Age_53 > 0.50 | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | |--- Age_55 > 0.50 | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | |--- Age_63 > 0.50 | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | |--- ZIPCode > 95025.50 | | | | | |--- weights: [0.60, 0.00] class: 0 | | | |--- Family_2 > 0.50 | | | | |--- Age_37 <= 0.50 | | | | | |--- weights: [2.20, 0.00] class: 0 | | | | |--- Age_37 > 0.50 | | | | | |--- CCAvg <= 1.25 | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | |--- CCAvg > 1.25 | | | | | | |--- weights: [0.10, 0.00] class: 0 | |--- CD_Account_1 > 0.50 | | |--- Securities_Account_1 <= 0.50 | | | |--- ZIPCode <= 95338.00 | | | | |--- CCAvg <= 0.15 | | | | | |--- weights: [0.40, 0.00] class: 0 | | | | |--- CCAvg > 0.15 | | | | | |--- CCAvg <= 2.63 | | | | | | |--- Age_52 <= 0.50 | | | | | | | |--- Age_57 <= 0.50 | | | | | | | | |--- Age_49 <= 0.50 | | | | | | | | | |--- Age_31 <= 0.50 | | | | | | | | | | |--- Age_53 <= 0.50 | | | | | | | | | | | |--- truncated branch of depth 10 | | | | | | | | | | |--- Age_53 > 0.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | | |--- Age_31 > 0.50 | | | | | | | | | | |--- ZIPCode <= 93594.00 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | | |--- ZIPCode > 93594.00 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | |--- Age_49 > 0.50 | | | | | | | | | |--- ZIPCode <= 90228.50 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- ZIPCode > 90228.50 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | |--- Age_57 > 0.50 | | | | | | | | |--- Education_3 <= 0.50 | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | |--- Education_3 > 0.50 | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | |--- Age_52 > 0.50 | | | | | | | |--- ZIPCode <= 92402.50 | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | |--- ZIPCode > 92402.50 | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | |--- CCAvg > 2.63 | | | | | | |--- Family_3 <= 0.50 | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | |--- Family_3 > 0.50 | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | |--- ZIPCode > 95338.00 | | | | |--- Education_3 <= 0.50 | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | |--- Education_3 > 0.50 | | | | | |--- weights: [0.20, 0.00] class: 0 | | |--- Securities_Account_1 > 0.50 | | | |--- Age_37 <= 0.50 | | | | |--- CCAvg <= 2.55 | | | | | |--- Mortgage <= 339.50 | | | | | | |--- Age_65 <= 0.50 | | | | | | | |--- Age_34 <= 0.50 | | | | | | | | |--- weights: [5.50, 0.00] class: 0 | | | | | | | |--- Age_34 > 0.50 | | | | | | | | |--- ZIPCode <= 94078.50 | | | | | | | | | |--- CCAvg <= 1.30 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- CCAvg > 1.30 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | |--- ZIPCode > 94078.50 | | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | |--- Age_65 > 0.50 | | | | | | | |--- Online_1 <= 0.50 | | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | | | |--- Online_1 > 0.50 | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | |--- Mortgage > 339.50 | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | |--- CCAvg > 2.55 | | | | | |--- Experience <= 26.00 | | | | | | |--- weights: [0.50, 0.00] class: 0 | | | | | |--- Experience > 26.00 | | | | | | |--- Education_2 <= 0.50 | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | |--- Education_2 > 0.50 | | | | | | | |--- weights: [0.00, 2.70] class: 1 | | | |--- Age_37 > 0.50 | | | | |--- weights: [0.00, 1.80] class: 1 |--- CCAvg > 2.85 | |--- Education_2 <= 0.50 | | |--- Education_3 <= 0.50 | | | |--- Family_3 <= 0.50 | | | | |--- Family_4 <= 0.50 | | | | | |--- ZIPCode <= 92566.50 | | | | | | |--- CCAvg <= 4.65 | | | | | | | |--- Experience <= 17.50 | | | | | | | | |--- weights: [3.30, 0.00] class: 0 | | | | | | | |--- Experience > 17.50 | | | | | | | | |--- CCAvg <= 3.00 | | | | | | | | | |--- weights: [1.30, 0.00] class: 0 | | | | | | | | |--- CCAvg > 3.00 | | | | | | | | | |--- Family_2 <= 0.50 | | | | | | | | | | |--- Age_64 <= 0.50 | | | | | | | | | | | |--- truncated branch of depth 6 | | | | | | | | | | |--- Age_64 > 0.50 | | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | | |--- Family_2 > 0.50 | | | | | | | | | | |--- CCAvg <= 3.35 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | | | |--- CCAvg > 3.35 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | |--- CCAvg > 4.65 | | | | | | | |--- Age_39 <= 0.50 | | | | | | | | |--- weights: [6.20, 0.00] class: 0 | | | | | | | |--- Age_39 > 0.50 | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | |--- ZIPCode > 92566.50 | | | | | | |--- weights: [17.80, 0.00] class: 0 | | | | |--- Family_4 > 0.50 | | | | | |--- CCAvg <= 2.95 | | | | | | |--- weights: [0.80, 0.00] class: 0 | | | | | |--- CCAvg > 2.95 | | | | | | |--- Age_30 <= 0.50 | | | | | | | |--- Age_50 <= 0.50 | | | | | | | | |--- Age_33 <= 0.50 | | | | | | | | | |--- Age_54 <= 0.50 | | | | | | | | | | |--- Age_35 <= 0.50 | | | | | | | | | | | |--- truncated branch of depth 7 | | | | | | | | | | |--- Age_35 > 0.50 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- Age_54 > 0.50 | | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | |--- Age_33 > 0.50 | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | |--- Age_50 > 0.50 | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | |--- Age_30 > 0.50 | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | |--- Family_3 > 0.50 | | | | |--- Experience <= 27.50 | | | | | |--- CCAvg <= 3.15 | | | | | | |--- Experience <= 16.50 | | | | | | | |--- Experience <= 14.50 | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | |--- Experience > 14.50 | | | | | | | | |--- weights: [0.50, 0.00] class: 0 | | | | | | |--- Experience > 16.50 | | | | | | | |--- weights: [0.00, 0.90] class: 1 | | | | | |--- CCAvg > 3.15 | | | | | | |--- weights: [0.00, 18.90] class: 1 | | | | |--- Experience > 27.50 | | | | | |--- Experience <= 31.50 | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | |--- Experience > 31.50 | | | | | | |--- weights: [0.50, 0.00] class: 0 | | |--- Education_3 > 0.50 | | | |--- CCAvg <= 4.55 | | | | |--- CCAvg <= 4.40 | | | | | |--- Age_41 <= 0.50 | | | | | | |--- ZIPCode <= 90041.50 | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | |--- ZIPCode > 90041.50 | | | | | | | |--- Age_43 <= 0.50 | | | | | | | | |--- Age_32 <= 0.50 | | | | | | | | | |--- Experience <= 30.50 | | | | | | | | | | |--- Age_56 <= 0.50 | | | | | | | | | | | |--- truncated branch of depth 13 | | | | | | | | | | |--- Age_56 > 0.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | | |--- Experience > 30.50 | | | | | | | | | | |--- Age_56 <= 0.50 | | | | | | | | | | | |--- weights: [0.00, 12.60] class: 1 | | | | | | | | | | |--- Age_56 > 0.50 | | | | | | | | | | | |--- truncated branch of depth 2 | | | | | | | | |--- Age_32 > 0.50 | | | | | | | | | |--- ZIPCode <= 94884.00 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- ZIPCode > 94884.00 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | |--- Age_43 > 0.50 | | | | | | | | |--- Securities_Account_1 <= 0.50 | | | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | | | | | |--- Securities_Account_1 > 0.50 | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | |--- Age_41 > 0.50 | | | | | | |--- Online_1 <= 0.50 | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | |--- Online_1 > 0.50 | | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | | |--- CCAvg > 4.40 | | | | | |--- Family_2 <= 0.50 | | | | | | |--- weights: [0.00, 1.80] class: 1 | | | | | |--- Family_2 > 0.50 | | | | | | |--- Securities_Account_1 <= 0.50 | | | | | | | |--- weights: [1.30, 0.00] class: 0 | | | | | | |--- Securities_Account_1 > 0.50 | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | |--- CCAvg > 4.55 | | | | |--- Age_62 <= 0.50 | | | | | |--- weights: [0.00, 47.70] class: 1 | | | | |--- Age_62 > 0.50 | | | | | |--- weights: [0.00, 1.80] class: 1 | |--- Education_2 > 0.50 | | |--- Age_64 <= 0.50 | | | |--- CCAvg <= 3.85 | | | | |--- Experience <= 29.50 | | | | | |--- Age_52 <= 0.50 | | | | | | |--- Age_40 <= 0.50 | | | | | | | |--- CCAvg <= 3.55 | | | | | | | | |--- Experience <= 27.50 | | | | | | | | | |--- Age_50 <= 0.50 | | | | | | | | | | |--- weights: [0.00, 15.30] class: 1 | | | | | | | | | |--- Age_50 > 0.50 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | |--- Experience > 27.50 | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | |--- CCAvg > 3.55 | | | | | | | | |--- Age_38 <= 0.50 | | | | | | | | | |--- Age_36 <= 0.50 | | | | | | | | | | |--- Age_37 <= 0.50 | | | | | | | | | | | |--- truncated branch of depth 4 | | | | | | | | | | |--- Age_37 > 0.50 | | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | | |--- Age_36 > 0.50 | | | | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | | | | |--- Age_38 > 0.50 | | | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | |--- Age_40 > 0.50 | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | | |--- Age_52 > 0.50 | | | | | | |--- Experience <= 26.50 | | | | | | | |--- weights: [0.30, 0.00] class: 0 | | | | | | |--- Experience > 26.50 | | | | | | | |--- weights: [0.10, 0.00] class: 0 | | | | |--- Experience > 29.50 | | | | | |--- Age_55 <= 0.50 | | | | | | |--- weights: [0.40, 0.00] class: 0 | | | | | |--- Age_55 > 0.50 | | | | | | |--- weights: [0.20, 0.00] class: 0 | | | |--- CCAvg > 3.85 | | | | |--- weights: [0.00, 65.70] class: 1 | | |--- Age_64 > 0.50 | | | |--- weights: [0.20, 0.00] class: 0
print (pd.DataFrame(model.feature_importances_, columns = ["Imp"], index = X_train.columns).sort_values(by = 'Imp', ascending = False))
Imp CCAvg 4.638885e-01 ZIPCode 6.618807e-02 CD_Account_1 5.913420e-02 Education_3 5.673423e-02 Experience 5.289057e-02 Mortgage 4.683768e-02 Family_4 3.865759e-02 Family_3 3.547004e-02 Family_2 2.768380e-02 Education_2 1.872774e-02 Age_26 9.687036e-03 Age_50 7.963646e-03 Age_53 7.494160e-03 Age_57 7.251624e-03 Age_37 7.203089e-03 Age_64 7.138749e-03 Age_52 6.972160e-03 Securities_Account_1 6.548680e-03 CreditCard_1 5.909079e-03 Age_30 5.592634e-03 Age_60 5.376324e-03 Age_31 5.361996e-03 Age_43 5.000970e-03 Age_34 3.996246e-03 Age_38 3.952983e-03 Age_54 3.922938e-03 Age_65 3.902376e-03 Age_28 3.426951e-03 Age_49 2.957346e-03 Online_1 2.813087e-03 Age_55 2.660605e-03 Age_35 2.532018e-03 Age_41 2.183722e-03 Age_33 1.796176e-03 Age_45 1.723536e-03 Age_32 1.717265e-03 Age_40 1.684690e-03 Age_39 1.673501e-03 Age_63 1.212901e-03 Age_36 1.135113e-03 Age_56 1.056293e-03 Age_51 6.792191e-04 Age_29 6.253830e-04 Age_44 5.744150e-04 Age_58 6.063270e-05 Age_62 3.221062e-16 Age_48 7.591728e-19 Age_59 0.000000e+00 Age_61 0.000000e+00 Age_46 0.000000e+00 Age_66 0.000000e+00 Age_67 0.000000e+00 Age_47 0.000000e+00 Age_27 0.000000e+00 Age_25 0.000000e+00 Age_24 0.000000e+00 Age_42 0.000000e+00
importances = model.feature_importances_
indices = np.argsort(importances)
plt.figure(figsize=(12,12))
plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='violet', align='center')
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel('Relative Importance')
plt.show()
from sklearn.model_selection import GridSearchCV
# Choose the type of classifier.
estimator = DecisionTreeClassifier(random_state=1,class_weight = {0:.10,1:.90}
)
# Grid of parameters to choose from
parameters = {
'max_depth': np.arange(1,10),
'criterion': ['entropy','gini'],
'splitter': ['best','random'],
'min_impurity_decrease': [0.000001,0.00001,0.0001],
'max_features': ['log2','sqrt']
}
# Type of scoring used to compare parameter combinations
scorer = metrics.make_scorer(metrics.recall_score)
# Run the grid search
grid_obj = GridSearchCV(estimator, parameters, scoring=scorer,cv=5)
grid_obj = grid_obj.fit(X_train, y_train)
# Set the clf to the best combination of parameters
estimator = grid_obj.best_estimator_
# Fit the best algorithm to the data.
estimator.fit(X_train, y_train)
DecisionTreeClassifier(class_weight={0: 0.1, 1: 0.9}, criterion='entropy',
max_depth=3, max_features='sqrt',
min_impurity_decrease=1e-06, random_state=1,
splitter='random')
make_confusion_matrix(estimator,y_test)
get_recall_score(estimator)
Recall on training set : 0.9516616314199395 Recall on test set : 0.8859060402684564
For my recall sroce I got a training sroce of .95 and and test sroce of .89 which is good and a whole lot better then my first model. I think this model can still be improved on by Cost Complexity Pruning
CCAvg has a a strong relationship with the dependent variable
plt.figure(figsize=(15,10))
out = tree.plot_tree(estimator,feature_names=feature_names,filled=True,fontsize=9,node_ids=False,class_names=None)
for o in out:
arrow = o.arrow_patch
if arrow is not None:
arrow.set_edgecolor('black')
arrow.set_linewidth(1)
plt.show()
# Text report showing the rules of a decision tree -
print(tree.export_text(estimator,feature_names=feature_names,show_weights=True))
|--- Age_25 <= 0.02 | |--- CD_Account_1 <= 0.60 | | |--- CCAvg <= 1.00 | | | |--- weights: [124.70, 14.40] class: 0 | | |--- CCAvg > 1.00 | | | |--- weights: [176.30, 190.80] class: 1 | |--- CD_Account_1 > 0.60 | | |--- Securities_Account_1 <= 0.17 | | | |--- weights: [4.40, 60.30] class: 1 | | |--- Securities_Account_1 > 0.17 | | | |--- weights: [7.10, 32.40] class: 1 |--- Age_25 > 0.02 | |--- weights: [4.40, 0.00] class: 0
print (pd.DataFrame(estimator.feature_importances_, columns = ["Imp"], index = X_train.columns).sort_values(by = 'Imp', ascending = False))
Imp CD_Account_1 0.495871 CCAvg 0.455215 Age_25 0.032307 Securities_Account_1 0.016608 Experience 0.000000 Age_59 0.000000 Age_49 0.000000 Age_50 0.000000 Age_51 0.000000 Age_52 0.000000 Age_53 0.000000 Age_54 0.000000 Age_55 0.000000 Age_56 0.000000 Age_57 0.000000 Age_58 0.000000 Age_62 0.000000 Age_60 0.000000 Age_61 0.000000 Age_47 0.000000 Age_63 0.000000 Age_64 0.000000 Age_65 0.000000 Age_66 0.000000 Age_67 0.000000 Education_2 0.000000 Education_3 0.000000 Online_1 0.000000 Age_48 0.000000 Age_45 0.000000 Age_46 0.000000 Age_32 0.000000 Mortgage 0.000000 Family_2 0.000000 Family_3 0.000000 Family_4 0.000000 Age_24 0.000000 Age_26 0.000000 Age_27 0.000000 Age_28 0.000000 Age_29 0.000000 Age_30 0.000000 Age_31 0.000000 Age_33 0.000000 ZIPCode 0.000000 Age_34 0.000000 Age_35 0.000000 Age_36 0.000000 Age_37 0.000000 Age_38 0.000000 Age_39 0.000000 Age_40 0.000000 Age_41 0.000000 Age_42 0.000000 Age_43 0.000000 Age_44 0.000000 CreditCard_1 0.000000
importances = model.feature_importances_
indices = np.argsort(importances)
plt.figure(figsize=(12,12))
plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='violet', align='center')
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel('Relative Importance')
plt.show()
clf = DecisionTreeClassifier(random_state=1,class_weight = {0:0.10,1:0.90})
path = clf.cost_complexity_pruning_path(X_train, y_train)
ccp_alphas, impurities = path.ccp_alphas, path.impurities
pd.DataFrame(path)
| ccp_alphas | impurities | |
|---|---|---|
| 0 | 0.000000e+00 | -7.164622e-16 |
| 1 | 1.805828e-19 | -7.162817e-16 |
| 2 | 1.805828e-19 | -7.161011e-16 |
| 3 | 1.805828e-19 | -7.159205e-16 |
| 4 | 1.805828e-19 | -7.157399e-16 |
| 5 | 1.805828e-19 | -7.155593e-16 |
| 6 | 1.805828e-19 | -7.153787e-16 |
| 7 | 3.611656e-19 | -7.150176e-16 |
| 8 | 3.611656e-19 | -7.146564e-16 |
| 9 | 3.792239e-19 | -7.142772e-16 |
| 10 | 3.792239e-19 | -7.138980e-16 |
| 11 | 3.792239e-19 | -7.135187e-16 |
| 12 | 3.792239e-19 | -7.131395e-16 |
| 13 | 5.056318e-19 | -7.126339e-16 |
| 14 | 5.056318e-19 | -7.121283e-16 |
| 15 | 5.959232e-19 | -7.115323e-16 |
| 16 | 6.500981e-19 | -7.108822e-16 |
| 17 | 9.209723e-19 | -7.099613e-16 |
| 18 | 1.011264e-18 | -7.089500e-16 |
| 19 | 1.011264e-18 | -7.079387e-16 |
| 20 | 1.083497e-18 | -7.068552e-16 |
| 21 | 1.083497e-18 | -7.057717e-16 |
| 22 | 1.083497e-18 | -7.046882e-16 |
| 23 | 1.083497e-18 | -7.036047e-16 |
| 24 | 1.191846e-18 | -7.024129e-16 |
| 25 | 1.300196e-18 | -7.011127e-16 |
| 26 | 1.300196e-18 | -6.998125e-16 |
| 27 | 1.354371e-18 | -6.984581e-16 |
| 28 | 1.444662e-18 | -6.970135e-16 |
| 29 | 1.769711e-18 | -6.952438e-16 |
| 30 | 2.148935e-18 | -6.930948e-16 |
| 31 | 2.455926e-18 | -6.906389e-16 |
| 32 | 2.780975e-18 | -6.878579e-16 |
| 33 | 2.925441e-18 | -6.849325e-16 |
| 34 | 3.178257e-18 | -6.817542e-16 |
| 35 | 3.250490e-18 | -6.785037e-16 |
| 36 | 3.575539e-18 | -6.749282e-16 |
| 37 | 6.500981e-18 | -6.684272e-16 |
| 38 | 8.704091e-18 | -6.597231e-16 |
| 39 | 8.704091e-18 | -6.510190e-16 |
| 40 | 8.704091e-18 | -6.423149e-16 |
| 41 | 8.704091e-18 | -6.336109e-16 |
| 42 | 8.704091e-18 | -6.249068e-16 |
| 43 | 8.704091e-18 | -6.162027e-16 |
| 44 | 9.426422e-18 | -6.067763e-16 |
| 45 | 1.300196e-17 | -5.937743e-16 |
| 46 | 1.300196e-17 | -5.807723e-16 |
| 47 | 1.413963e-17 | -5.666327e-16 |
| 48 | 1.739012e-17 | -5.492426e-16 |
| 49 | 1.769711e-17 | -5.315455e-16 |
| 50 | 2.166994e-17 | -5.098755e-16 |
| 51 | 2.600392e-17 | -4.838716e-16 |
| 52 | 2.752082e-17 | -4.563508e-16 |
| 53 | 1.196180e-16 | -3.367327e-16 |
| 54 | 1.608993e-16 | -1.758335e-16 |
| 55 | 2.309473e-16 | 5.511387e-17 |
| 56 | 1.540938e-04 | 3.081875e-04 |
| 57 | 1.591186e-04 | 6.264246e-04 |
| 58 | 1.602800e-04 | 1.267545e-03 |
| 59 | 1.614585e-04 | 1.590462e-03 |
| 60 | 1.615328e-04 | 1.913527e-03 |
| 61 | 1.615328e-04 | 2.236593e-03 |
| 62 | 2.833337e-04 | 4.503262e-03 |
| 63 | 2.926163e-04 | 5.673728e-03 |
| 64 | 2.927781e-04 | 6.259284e-03 |
| 65 | 2.927781e-04 | 6.844840e-03 |
| 66 | 2.927781e-04 | 7.137618e-03 |
| 67 | 2.927781e-04 | 7.430396e-03 |
| 68 | 2.927781e-04 | 7.723175e-03 |
| 69 | 2.927781e-04 | 8.015953e-03 |
| 70 | 2.927781e-04 | 8.308731e-03 |
| 71 | 2.927781e-04 | 8.601509e-03 |
| 72 | 2.927781e-04 | 8.894287e-03 |
| 73 | 2.927781e-04 | 9.187065e-03 |
| 74 | 2.927781e-04 | 9.479843e-03 |
| 75 | 2.987532e-04 | 1.067486e-02 |
| 76 | 3.013893e-04 | 1.218180e-02 |
| 77 | 3.069192e-04 | 1.340948e-02 |
| 78 | 3.072908e-04 | 1.586781e-02 |
| 79 | 3.081875e-04 | 1.617599e-02 |
| 80 | 3.081875e-04 | 1.648418e-02 |
| 81 | 3.136909e-04 | 1.679787e-02 |
| 82 | 3.136909e-04 | 1.711156e-02 |
| 83 | 3.165466e-04 | 1.774466e-02 |
| 84 | 3.193943e-04 | 1.806405e-02 |
| 85 | 3.211115e-04 | 1.870627e-02 |
| 86 | 4.198187e-04 | 1.996573e-02 |
| 87 | 4.636369e-04 | 2.089300e-02 |
| 88 | 5.271471e-04 | 2.142015e-02 |
| 89 | 5.283215e-04 | 2.194847e-02 |
| 90 | 5.323239e-04 | 2.248080e-02 |
| 91 | 5.323239e-04 | 2.301312e-02 |
| 92 | 5.562785e-04 | 2.579451e-02 |
| 93 | 5.720806e-04 | 2.751075e-02 |
| 94 | 5.735167e-04 | 3.726054e-02 |
| 95 | 5.855563e-04 | 3.784609e-02 |
| 96 | 6.249020e-04 | 3.847100e-02 |
| 97 | 6.632806e-04 | 4.046084e-02 |
| 98 | 7.319453e-04 | 4.119278e-02 |
| 99 | 7.319453e-04 | 4.192473e-02 |
| 100 | 7.319453e-04 | 4.265667e-02 |
| 101 | 7.528581e-04 | 4.340953e-02 |
| 102 | 7.533065e-04 | 4.416284e-02 |
| 103 | 7.647856e-04 | 4.645719e-02 |
| 104 | 7.782134e-04 | 4.723541e-02 |
| 105 | 7.842272e-04 | 5.037232e-02 |
| 106 | 8.030242e-04 | 5.278139e-02 |
| 107 | 8.066336e-04 | 5.358802e-02 |
| 108 | 8.090703e-04 | 5.601523e-02 |
| 109 | 9.595708e-04 | 6.177266e-02 |
| 110 | 9.647609e-04 | 6.273742e-02 |
| 111 | 9.647609e-04 | 6.370218e-02 |
| 112 | 9.701438e-04 | 6.758276e-02 |
| 113 | 1.000748e-03 | 6.858350e-02 |
| 114 | 1.025315e-03 | 7.165945e-02 |
| 115 | 1.039970e-03 | 7.373939e-02 |
| 116 | 1.054001e-03 | 7.479339e-02 |
| 117 | 1.099703e-03 | 7.699280e-02 |
| 118 | 1.141835e-03 | 7.813463e-02 |
| 119 | 1.195201e-03 | 7.932983e-02 |
| 120 | 1.203002e-03 | 8.654784e-02 |
| 121 | 1.216155e-03 | 8.776400e-02 |
| 122 | 1.283281e-03 | 8.904728e-02 |
| 123 | 1.441643e-03 | 9.048892e-02 |
| 124 | 1.463891e-03 | 9.195281e-02 |
| 125 | 1.505716e-03 | 9.345853e-02 |
| 126 | 1.627718e-03 | 9.834168e-02 |
| 127 | 1.636215e-03 | 1.130676e-01 |
| 128 | 1.637329e-03 | 1.228916e-01 |
| 129 | 1.689809e-03 | 1.245814e-01 |
| 130 | 1.720032e-03 | 1.331816e-01 |
| 131 | 1.769055e-03 | 1.349506e-01 |
| 132 | 1.811565e-03 | 1.367622e-01 |
| 133 | 1.854098e-03 | 1.423245e-01 |
| 134 | 1.897997e-03 | 1.442225e-01 |
| 135 | 1.929846e-03 | 1.480822e-01 |
| 136 | 1.951854e-03 | 1.500340e-01 |
| 137 | 1.952976e-03 | 1.519870e-01 |
| 138 | 2.001268e-03 | 1.539883e-01 |
| 139 | 2.110332e-03 | 1.560986e-01 |
| 140 | 2.124595e-03 | 1.582232e-01 |
| 141 | 2.269485e-03 | 1.604927e-01 |
| 142 | 2.284272e-03 | 1.627770e-01 |
| 143 | 2.565682e-03 | 1.679083e-01 |
| 144 | 2.659040e-03 | 1.705674e-01 |
| 145 | 2.685595e-03 | 1.732530e-01 |
| 146 | 3.835456e-03 | 1.809239e-01 |
| 147 | 3.940450e-03 | 1.848643e-01 |
| 148 | 4.300942e-03 | 2.063690e-01 |
| 149 | 4.423269e-03 | 2.107923e-01 |
| 150 | 1.526260e-02 | 2.718427e-01 |
| 151 | 1.599689e-02 | 2.878396e-01 |
| 152 | 2.953886e-02 | 3.173784e-01 |
| 153 | 1.821440e-01 | 4.995225e-01 |
fig, ax = plt.subplots(figsize=(10,5))
ax.plot(ccp_alphas[:-1], impurities[:-1], marker='o', drawstyle="steps-post")
ax.set_xlabel("effective alpha")
ax.set_ylabel("total impurity of leaves")
ax.set_title("Total Impurity vs effective alpha for training set")
plt.show()
clfs = []
for ccp_alpha in ccp_alphas:
clf = DecisionTreeClassifier(random_state=1, ccp_alpha=ccp_alpha,class_weight = {0:0.10,1:0.90})
clf.fit(X_train, y_train)
clfs.append(clf)
print("Number of nodes in the last tree is: {} with ccp_alpha: {}".format(
clfs[-1].tree_.node_count, ccp_alphas[-1]))
Number of nodes in the last tree is: 1 with ccp_alpha: 0.18214402026679288
clfs = clfs[:-1]
ccp_alphas = ccp_alphas[:-1]
node_counts = [clf.tree_.node_count for clf in clfs]
depth = [clf.tree_.max_depth for clf in clfs]
fig, ax = plt.subplots(2, 1,figsize=(10,7))
ax[0].plot(ccp_alphas, node_counts, marker='o', drawstyle="steps-post")
ax[0].set_xlabel("alpha")
ax[0].set_ylabel("number of nodes")
ax[0].set_title("Number of nodes vs alpha")
ax[1].plot(ccp_alphas, depth, marker='o', drawstyle="steps-post")
ax[1].set_xlabel("alpha")
ax[1].set_ylabel("depth of tree")
ax[1].set_title("Depth vs alpha")
fig.tight_layout()
recall_train=[]
for clf in clfs:
pred_train3=clf.predict(X_train)
values_train=metrics.recall_score(y_train,pred_train3)
recall_train.append(values_train)
recall_test=[]
for clf in clfs:
pred_test3=clf.predict(X_test)
values_test=metrics.recall_score(y_test,pred_test3)
recall_test.append(values_test)
train_scores = [clf.score(X_train, y_train) for clf in clfs]
test_scores = [clf.score(X_test, y_test) for clf in clfs]
fig, ax = plt.subplots(figsize=(15,5))
ax.set_xlabel("alpha")
ax.set_ylabel("Recall")
ax.set_title("Recall vs alpha for training and testing sets")
ax.plot(ccp_alphas, recall_train, marker='o', label="train",
drawstyle="steps-post",)
ax.plot(ccp_alphas, recall_test, marker='o', label="test",
drawstyle="steps-post")
ax.legend()
plt.show()
# creating the model where we get highest train and test recall
index_best_model = np.argmax(recall_test)
best_model = clfs[index_best_model]
print(best_model)
DecisionTreeClassifier(ccp_alpha=0.002284272280085535,
class_weight={0: 0.1, 1: 0.9}, random_state=1)
best_model.fit(X_train, y_train)
DecisionTreeClassifier(ccp_alpha=0.002284272280085535,
class_weight={0: 0.1, 1: 0.9}, random_state=1)
make_confusion_matrix(best_model,y_test)
get_recall_score(best_model)
Recall on training set : 0.9154078549848943 Recall on test set : 0.8053691275167785
plt.figure(figsize=(5,5))
out = tree.plot_tree(best_model,feature_names=feature_names,filled=True,fontsize=9,node_ids=False,class_names=None)
for o in out:
arrow = o.arrow_patch
if arrow is not None:
arrow.set_edgecolor('black')
arrow.set_linewidth(1)
plt.show()
best_model = DecisionTreeClassifier(ccp_alpha=0.0022,class_weight={0: 0.10, 1: 0.90},random_state=1)
best_model.fit(X_train, y_train)
DecisionTreeClassifier(ccp_alpha=0.0022, class_weight={0: 0.1, 1: 0.9},
random_state=1)
make_confusion_matrix(best_model,y_test)
get_recall_score(best_model)
Recall on training set : 0.9154078549848943 Recall on test set : 0.7986577181208053
plt.figure(figsize=(15,10))
out = tree.plot_tree(best_model,feature_names=feature_names,filled=True,fontsize=9,node_ids=False,class_names=None)
for o in out:
arrow = o.arrow_patch
if arrow is not None:
arrow.set_edgecolor('black')
arrow.set_linewidth(1)
plt.show()
print (pd.DataFrame(best_model.feature_importances_, columns = ["Imp"], index = X_train.columns).sort_values(by = 'Imp', ascending = False))
Imp CCAvg 0.592505 CD_Account_1 0.086548 Education_3 0.070885 Family_4 0.046926 Mortgage 0.046871 Family_3 0.044140 Education_2 0.024073 Family_2 0.019914 ZIPCode 0.018325 Experience 0.017946 Age_57 0.008255 Age_26 0.007953 Age_60 0.007869 Securities_Account_1 0.007791 Age_52 0.000000 Age_58 0.000000 Age_53 0.000000 Age_54 0.000000 Age_55 0.000000 Age_56 0.000000 Age_51 0.000000 Age_64 0.000000 Age_59 0.000000 Age_61 0.000000 Age_62 0.000000 Age_63 0.000000 Age_49 0.000000 Age_65 0.000000 Age_66 0.000000 Age_67 0.000000 Online_1 0.000000 Age_50 0.000000 Age_45 0.000000 Age_48 0.000000 Age_47 0.000000 Age_24 0.000000 Age_25 0.000000 Age_27 0.000000 Age_28 0.000000 Age_29 0.000000 Age_30 0.000000 Age_31 0.000000 Age_32 0.000000 Age_33 0.000000 Age_34 0.000000 Age_35 0.000000 Age_36 0.000000 Age_37 0.000000 Age_38 0.000000 Age_39 0.000000 Age_40 0.000000 Age_41 0.000000 Age_42 0.000000 Age_43 0.000000 Age_44 0.000000 Age_46 0.000000 CreditCard_1 0.000000
importances = best_model.feature_importances_
indices = np.argsort(importances)
plt.figure(figsize=(12,12))
plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='violet', align='center')
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel('Relative Importance')
plt.show()
Creating model with 0.0032 ccp_alpha
best_model2 = DecisionTreeClassifier(ccp_alpha=0.0032,
class_weight={0: 0.10, 1: 0.90},random_state=1)
best_model2.fit(X_train, y_train)
DecisionTreeClassifier(ccp_alpha=0.0032, class_weight={0: 0.1, 1: 0.9},
random_state=1)
make_confusion_matrix(best_model2,y_test)
get_recall_score(best_model2)
Recall on training set : 0.9033232628398792 Recall on test set : 0.8053691275167785
plt.figure(figsize=(15,10))
out = tree.plot_tree(best_model2,feature_names=feature_names,filled=True,fontsize=9,node_ids=False,class_names=None)
for o in out:
arrow = o.arrow_patch
if arrow is not None:
arrow.set_edgecolor('black')
arrow.set_linewidth(1)
plt.show()
print(tree.export_text(best_model2,feature_names=feature_names,show_weights=True))
|--- CCAvg <= 2.85 | |--- CD_Account_1 <= 0.50 | | |--- Mortgage <= 281.00 | | | |--- CCAvg <= 1.05 | | | | |--- weights: [124.20, 9.00] class: 0 | | | |--- CCAvg > 1.05 | | | | |--- Age_26 <= 0.50 | | | | | |--- Education_3 <= 0.50 | | | | | | |--- weights: [92.20, 15.30] class: 0 | | | | | |--- Education_3 > 0.50 | | | | | | |--- CCAvg <= 2.35 | | | | | | | |--- CCAvg <= 1.35 | | | | | | | | |--- Family_2 <= 0.50 | | | | | | | | | |--- weights: [3.90, 0.00] class: 0 | | | | | | | | |--- Family_2 > 0.50 | | | | | | | | | |--- weights: [0.00, 4.50] class: 1 | | | | | | | |--- CCAvg > 1.35 | | | | | | | | |--- weights: [32.80, 4.50] class: 0 | | | | | | |--- CCAvg > 2.35 | | | | | | | |--- weights: [4.10, 7.20] class: 1 | | | | |--- Age_26 > 0.50 | | | | | |--- weights: [1.50, 2.70] class: 1 | | |--- Mortgage > 281.00 | | | |--- weights: [6.20, 12.60] class: 1 | |--- CD_Account_1 > 0.50 | | |--- weights: [9.50, 24.30] class: 1 |--- CCAvg > 2.85 | |--- Education_2 <= 0.50 | | |--- Education_3 <= 0.50 | | | |--- Family_3 <= 0.50 | | | | |--- Family_4 <= 0.50 | | | | | |--- ZIPCode <= 92566.50 | | | | | | |--- CCAvg <= 4.65 | | | | | | | |--- Experience <= 17.50 | | | | | | | | |--- weights: [3.30, 0.00] class: 0 | | | | | | | |--- Experience > 17.50 | | | | | | | | |--- weights: [3.30, 7.20] class: 1 | | | | | | |--- CCAvg > 4.65 | | | | | | | |--- weights: [6.30, 0.00] class: 0 | | | | | |--- ZIPCode > 92566.50 | | | | | | |--- weights: [17.80, 0.00] class: 0 | | | | |--- Family_4 > 0.50 | | | | | |--- weights: [2.30, 13.50] class: 1 | | | |--- Family_3 > 0.50 | | | | |--- weights: [1.30, 19.80] class: 1 | | |--- Education_3 > 0.50 | | | |--- weights: [6.00, 91.80] class: 1 | |--- Education_2 > 0.50 | | |--- weights: [2.20, 85.50] class: 1
print (pd.DataFrame(best_model2.feature_importances_, columns = ["Imp"], index = X_train.columns).sort_values(by = 'Imp', ascending = False))
Imp CCAvg 0.612843 CD_Account_1 0.090535 Education_3 0.074150 Family_4 0.049088 Mortgage 0.049030 Family_3 0.046173 Education_2 0.025182 Family_2 0.020831 ZIPCode 0.012077 Experience 0.011771 Age_26 0.008319 Age_54 0.000000 Age_57 0.000000 Age_56 0.000000 Age_55 0.000000 Age_52 0.000000 Age_53 0.000000 Age_59 0.000000 Age_51 0.000000 Age_58 0.000000 Age_64 0.000000 Age_60 0.000000 Age_61 0.000000 Age_62 0.000000 Age_63 0.000000 Age_49 0.000000 Age_65 0.000000 Age_66 0.000000 Age_67 0.000000 Securities_Account_1 0.000000 Online_1 0.000000 Age_50 0.000000 Age_45 0.000000 Age_48 0.000000 Age_47 0.000000 Age_24 0.000000 Age_25 0.000000 Age_27 0.000000 Age_28 0.000000 Age_29 0.000000 Age_30 0.000000 Age_31 0.000000 Age_32 0.000000 Age_33 0.000000 Age_34 0.000000 Age_35 0.000000 Age_36 0.000000 Age_37 0.000000 Age_38 0.000000 Age_39 0.000000 Age_40 0.000000 Age_41 0.000000 Age_42 0.000000 Age_43 0.000000 Age_44 0.000000 Age_46 0.000000 CreditCard_1 0.000000
importances = best_model2.feature_importances_
indices = np.argsort(importances)
plt.figure(figsize=(12,12))
plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='violet', align='center')
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel('Relative Importance')
plt.show()
comparison_frame = pd.DataFrame({'Model':['Initial decision tree model','Decision treee with hyperparameter tuning',
'Decision tree with post-pruning','Best Model using ccp_alpha'], 'Train_Recall':[1,0.95,0.91,0.90], 'Test_Recall':[0.51,0.88,0.79,0.80]})
comparison_frame
| Model | Train_Recall | Test_Recall | |
|---|---|---|---|
| 0 | Initial decision tree model | 1.00 | 0.51 |
| 1 | Decision treee with hyperparameter tuning | 0.95 | 0.88 |
| 2 | Decision tree with post-pruning | 0.91 | 0.79 |
| 3 | Best Model using ccp_alpha | 0.90 | 0.80 |
I think my recall sroce was the best at Decision treee with hyperparameter tuning and i shouldnt have done post-pruning or Best Model using ccp_alpha.